Conf42 JavaScript 2023 - Online

Creating videos... with React!

Video size:

Abstract

I bet you’ve always used React to build websites and applications, right? What if I told you that we can also edit a movie with it! Learn how a simple render engine built in React and Node.js works to make videos using React components.

Summary

  • Today we will learn something about react node js and how we can create effective videos using just react. As a bonus, we will see how to create a scalable cloud based rendering farm as well.
  • Remotion is a react library to make videos programmatically. Possible use cases for the tool include parameterized videos and automated video workflows. You can also create cloud based videos.
  • Working with remotion means writing code to create our video. From here you can render your video to the composition. We can customize the render settings and when we are happy with the result, we will click on render videos. How can we render at scale?
  • Nearform is a services company. We are some of the major contributors to the Nodejs ecosystem. If you have any problem with your Nodejs application, just contact us. Thanks a lot for joining the session. It was really fun.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hey folks, welcome here. Thanks for joining this session. Today we will learn something about react node js and how we can create effective videos using just react. The first time I heard someone saying, hey, you know you can create a video with react, my reaction was like, what is this useful? Shall I use? It doesn't make any sense to me. But today we will actually see how we can create videos programmatically using react. As a bonus, we will see how to create a scalable cloud based rendering farm as well. Before we dive into the actual content, just a couple of words about me. I'm Alfonso. I'm a senior developer at Nearform. If youve want, just send me a connection on LinkedIn. I'm always more than happy to be connected with interesting people. Let's get started from the ground. What is a video? We can see a video like a sequence of images over time, usually third images or 6th images, really, depending on your video. And then of course we also have audio tracks. Let's ask ourselves for one moment, as a developer, am I able to create an image maybe by doing the screenshot of a viewport in HTML and make, I don't know, 30 or 60 screenshots per second, while the viewport is performing some action, like running transitions or playing a video. In that case, we obtain, for example, 30 images per second. We just need to find a way to stitch them together, add an audio on top, and we created our first video. So this is what we will see today. Usually editing tools are really complex, really complicated. They have tones of features. You can do virtually everything with those tools. But today we will do something way, way simpler. The main tool that we will use is remotion. Remotion is a react library to make videos programmatically. When you download remotion, you have a lot of tools. First of all, you have the remotion library, which expose a few abstractions that you can use to create your video. Then you have the remotion player, which is such a cool components because this means that thanks to this player, we can preview our video live without the need to render the video. So for example, we create an animation based on the color of something, and then we can preview the actual result without rendering to the user. Then we will also see remote Lambda. If you're not familiar with lambda, that's not a problem. Lambda is just a function as a service from AWS. This means that you can run your functions on the cloud and they will take care of all the server stuff. How does remotion work under the hood? First of all, we create our react code. So we create our video with code and we will see it in a few minutes. Then we use puppeteer to make screenshots of the viewports. And then we connect all those images using Ffmpeg and we attach the Audi as well. What are potential use cases for this tool? First of all, parameterized videos and we will see it soon. This means that we can have the same video and we just need to change a few parameters and the output will change. Then automated video workflows. Let's say that when we run a GitHub action or when something happens, we want to create our video based on some parameters that we pass and we send it on slack through email, we upload it on YouTube. We can do everything we want. Last but not least, cloud based videos. Sas, you can create your own tool to trim videos, adding subtitles. You can visually do everything with videos thanks to remotion. And then of course you can offer the tool you created on top of remotion to customers. Let's now see a quick example of what we can achieve using remotion. As you can see here we have a video, then we have this image, then we have this text. Everything is animated. We have this images slideshow of course also the images are animated as well. Now you cannot hear it, but there is this happy birthday song in the background. Then we have this text and then we have this fade effect. Of course this was a really simple example, but we can create videos as complex as we want. And here we have the source code of our video. If we analyze this source code and we will do it in a bit, we can see that it is just a simple JavaScript object. We have the base settings of our project, like the FPS, the name. Then we have the timeline. If you're familiar with tools like Davinci Resolve or Adobe Premiere Pro, you are probably used to the concept of the timeline. Then inside the timeline you can have multiple tracks, usually video tracks and audio tracks. Youve can have multiple video tracks and multiple audio tracks, and inside each track you can add multiple assets and this is exactly what we are doing here. Remotion as a tool is really quickly evolving and it is production ready. So if you want to start using it, you can just run NPM init video, you can take a look at the documentation and I promise you that you will have a lot of fun. Of course we can choose if we want to use remotion itself, or if we want to create another abstraction on top of remotion and that's what we will see today. In this schema we have a video editor tool that we can build, but it is out of scope. Then this video editing tool will create a JSON video description. In this case it was a JavaScript object, but we can use a simple JSON file. Then we will pass this JSON description to the render engine based on remotion, and as output we will get a video. All right, so it's time to code. Let's build our first render engine. If you are interested, by following this link here, you can clone the project and play with it. We are on vs code. I already started the project using NPM start and this started remotion studio. Before we look at the code, I want to spend just a couple of seconds on remotion studio because I think that is such a useful tool when youve work with remotion. We say that working with remotion means writing code to create our video. How can we know if our code will produce the video that we want? For this reason, remotion created Remotion Studio inside Remotion Studio you have this timeline and you can see your video. You can see the preview of your video. Then you can see all your compositions. In this case we have just one composition, and we will see what a composition is in a few moments. Then we have the props that we will see as well. And we can also render our video inside remotion. So we just need to click this render button. We can customize the render settings and when we are happy with the result, we will click on render videos, but back to the code. Let's see how our render engine works. It is actually really simple. When we start inside the root of the project, we can see the composition. The composition is one of the remotion abstractions. It represents the video that want to create. Inside the composition youve will put all the sequences, all the assets, and then from here you can render your video to the composition. We have to pass a components, a schema if we want, and some default props. Let's see the component. This is the timeline component that I created in this case. And as you can see, we have one main sequence. We can have multiple sequences. Of course, we are looping through the video tracks because we said before that. For example, in Adobe premiere Pro you can have multiple video tracks and multiple audio tracks as well. Then inside each audio track I have a sequence which contains the name of the audio track. And then for each asset I have a specific sequence, and inside the sequence I declare where the sequence should start and the length of the sequence in frames. Then inside the sequence I will add my audio. Of course, the sequence and the audio are remotion abstractions as well. If you want to learn more about that, just go into the remotion documentation and you will find everything youve need back on the timeline component. Let's take just a quick look to the video track. Here we have the same thing. So we have the sequence, then we have the sequence for each asset. And then in this case it's slightly more complex because we can have the asset type based on our premiere pro example. If you ever use the premiere pro in the video track, you can add videos, but you can add images and text as well. So that's exactly what we are doing here. We are adding video components, but we are adding text, we are adding images. And since you can literally add everything you want, we are also adding a CSS container so that you can import your own CSS. Now let's take just a quick look to the schema. If youve familiar with Zod. This is a Zod schema and it represents exactly what we saw before. In fact, if we go into remotion studio, in the props section, we can see that we can customize everything about our video. In theory, we can create new videos directly from this interface. If you have a simpler schema, you can even create your own video directly here. So let's say that you are a social media manager, and every time a football player makes a goal, you want to create a video for your social media. In that case, here you can say what is the current score, what is the player name, and all the information that you need. Then you click on render and your video is ready to bet uploaded on social media or whatever. I don't want to go too deep into the code because of course if you are interested, you can take a look on GitHub. But I'm interested in giving you the main ideas that youve need to use remotion back to our presentation. There is just one more thing that we need to address. So how can we render at scale right now? We saw that we can render one single video on our computer by installing remotion, and of course it works. But let's say that we need to produce a massive amount of videos. How can we do it? Well, in this case, remotion gives us multiple options. The first one is render locally, and the second one is rendering on Lambda. By render locally, we mean just rendering on a machine, which can be on premise or on cloud or even your own computer, of course if you need to render at scale. So if you have hundreds or thousands of job requests, then you will probably have a pool of machines that will pull your video jobs and you will have probably an auto scaling as well to manage load. The second option, which is really cool in my opinion, is rendering on lambda. This is really cheap, really simple to set up. If you have an AWS account, it just takes five minutes, more or less. And the results are great because using parallelization you can render a long video in minutes. One of the main limitations actually is that on lambda, at least for what the doc says, you can only render videos that are less than 30 minutes. That's a limitation of course. But what you can do, you can have renderer lambda for all the videos, and if the video is bigger than 30 minutes, then you can file over to a render locally on a machine. Let's quickly see on vs code how we can run remotion on lambda. As I said before, in this case I already made their setup and I will just show you the script to start the rendering. Here we are importing a bunch of things from remotion itself. Here I exported my video configuration, which in this case is the exact same video, but with Goku instead of SpongeBob. So we have these props that we are passing to the remotion lambda. Then we load some environment variables, then we get our functions. We get the functions that we deployed on AWS because we need the name of the function and then we just need to call this render media or lambda method. We will get in, output this render id and we can use the render id to evaluate what is the current status. So here we have a while true a simple loop and we will get the current progress. We will show the progress to the user and when render finished we will show the user the actual URL of our video. The video itself, once it's finished, will be stored on s three. Again, if you are not familiar with AWS, no worries. S three is just their storage system, so you can see it like a Google Drive or a Dropbox. And that's all for today. We covered a lot of topics. I just want to leave you with the challenge. So if you want, if you have the time, or if you wish to learn more about remotion, just check out the repo, customize the project or even the render engine itself and share with us. Youve first video. Before we jump off, just a couple of words about Nearform. Nearform is a services company. We are some of the major contributors to the Nodejs ecosystem. So if you have any problem with your Nodejs application, or if you want the support of our teams, just contact us. If you have any questions, feel free to contact me on my LinkedIn or on the discord server. Still, thanks a lot for joining the session. It was really fun.
...

Alfonso Graziano

Senior Software Developer @ NearForm

Alfonso Graziano's LinkedIn account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways