Serverless Smart Media Streaming Engine with AI/ML Backend
This demo shows how you can use CDK and amplify to build a streaming service with a smart backend that analyze the streaming media using a AI/ML service. In this particular demo, it sends a soccer game that Tottenham plays, generate clips, and services only clips that Son Heung-min appears.
- Elemental Media Services for streaming and vod contents
- Serverless Event-Driven Architecture
- All infra-structure resources as a code using CDK
- Simple React frontend using Amplify
- All code written in typescript
The streaming feature leverages Amazon Elemental Services. Amazon Elemental MediaLive provides live video processing service and Amazon Elemental MediaPackage provides packaging and distributions. And MediaPackage is integrated with the Amazon CloudFront to deliver live stream to viewers with low latency and high transfer speeds.
VoD Generation Backend with Rekognition
This demo shows a good demonstration on how you can use various types of events when developing serverless architecture. Amazon Elemental MediaLive has two output groups, one for MediaPackage for live stream and another for S3 Archive. The S3 Archive output group archives live videos into Amazon S3 bucket which becomes the first event of Event-Driven Serverless backend.
When a media file is uploaded to S3 Archive Bucket, a Lambda is invoked to put a message including the object information into SQS queue. Then another lambda is called via SQS target, to call Amazon Elemental MediaConvert to convert the file.
MediaConvert also has two output groups. One output group converts the file into HLS so it can be served as VoD Content. The other output group converts the file into MP4, because Amazon Rekognition can only analyze MP4 files.
Amazon EventBridge watches the jobs of MediaConvert, and it triggers an event when it completes and calls the third lambda. The third lambda calles Amazon Rekognition to analyze who appears in the video clip. And if a certain player is found (“Son Heung-min” in this demo), it puts the metadata in Amazon DynamoDB.
For ease of configuration and deployment, this demo uses AWS Amplify for the frontend and AWS CDK for backend. To keep the infrastructure management simple and consistent, all the resources are defined and deployed using AWS CDK. And the resource information is exported into a file in JSON format, and the frontend imports that file to configure AWS Amplify resources. Detailed instruction will be provided in the later chapter.
- aws account
- typescript >= 4.8
- aws-cdk >= 2.54
- node >= 16.17.0
- npm >= 8.15.0
- Download the code
git clone https://github.com/atheanchu/MediaReplayEngine.git
- Install Packages for CDK backend
- Export AWS CLI
- Bootstrap and synthesize CDK
cdk bootstrap cdk synth
- Deploy CDK – export the output to the output file in frontend directory.
cdk deploy -O ./frontend/src/cdk-exports.json
- Install packages for frontend React application
cd ./frontend npm install
Testing the demo
For detailed instructions on how to use OBS to stream your local video to MediaLive, please check this link.
Install Open Broadcaster Software(OBS)
Click Settings and Streams. Choose Custom for Service.
./frontend/cdk-exports.json. The value must be in the format of
rtmp://<ip address>:<port>/<stream key>. For Server, input
rtmp://<ip address>:<port>/and for Stream Key, put
<stream key>. For instance, if
- Server: rtmp://555.555.555.555:9999/ - Stream Key: streamkey
Play a video that Son Heung-min is in. It could be a local file or youtube video.
From Source, click
+button and choose Window Capture. Select the window you want to start streaming to MediaLive input and click the button
Run the frontend application
cd ./frontend npm start
- It will show the streaming video at the top of the page. It takes a few minutes for the backend to generate clips and analyze each clips which is 60 seconds long. Click the refresh icon at the bottom right corner, and all the clips that a certain celebrity is in will show up below.
- Stop Streaming from OBS by clicking button
- Stop MediaLive Channel by clicking
- Since all the resources are deployed using AWS CDK, you can clean up simply by running
cdk destroy. Although this demo uses AWS Amplify, we don’t have to run
amplify deletebecause no resources are created using through Amplify.
This sample code is made available under MIT-0 license. See the LICENSE file.