Serverless Smart Media Streaming Engine with AI/ML Backend
Overview
This demo shows how you can use CDK and amplify to build a streaming service with a smart backend that analyze the streaming media using a AI/ML service. In this particular demo, it sends a soccer game that Tottenham plays, generate clips, and services only clips that Son Heung-min appears.
- Elemental Media Services for streaming and vod contents
- Serverless Event-Driven Architecture
- All infra-structure resources as a code using CDK
- Simple React frontend using Amplify
- All code written in typescript
Architecture
Live Streaming
The streaming feature leverages Amazon Elemental Services. Amazon Elemental MediaLive provides live video processing service and Amazon Elemental MediaPackage provides packaging and distributions. And MediaPackage is integrated with the Amazon CloudFront to deliver live stream to viewers with low latency and high transfer speeds.
VoD Generation Backend with Rekognition
This demo shows a good demonstration on how you can use various types of events when developing serverless architecture. Amazon Elemental MediaLive has two output groups, one for MediaPackage for live stream and another for S3 Archive. The S3 Archive output group archives live videos into Amazon S3 bucket which becomes the first event of Event-Driven Serverless backend.
When a media file is uploaded to S3 Archive Bucket, a Lambda is invoked to put a message including the object information into SQS queue. Then another lambda is called via SQS target, to call Amazon Elemental MediaConvert to convert the file.
MediaConvert also has two output groups. One output group converts the file into HLS so it can be served as VoD Content. The other output group converts the file into MP4, because Amazon Rekognition can only analyze MP4 files.
Amazon EventBridge watches the jobs of MediaConvert, and it triggers an event when it completes and calls the third lambda. The third lambda calles Amazon Rekognition to analyze who appears in the video clip. And if a certain player is found (“Son Heung-min” in this demo), it puts the metadata in Amazon DynamoDB.
Frontend Integration
For ease of configuration and deployment, this demo uses AWS Amplify for the frontend and AWS CDK for backend. To keep the infrastructure management simple and consistent, all the resources are defined and deployed using AWS CDK. And the resource information is exported into a file in JSON format, and the frontend imports that file to configure AWS Amplify resources. Detailed instruction will be provided in the later chapter.
Prerequisites
- aws account
- aws-cli
- typescript >= 4.8
- aws-cdk >= 2.54
- node >= 16.17.0
- npm >= 8.15.0
- docker
- git
Installation
- Download the code
git clone https://github.com/atheanchu/MediaReplayEngine.git
- Install Packages for CDK backend
npm install
- Export AWS CLI
- Bootstrap and synthesize CDK
cdk bootstrap
cdk synth
- Deploy CDK – export the output to the output file in frontend directory.
cdk deploy -O ./frontend/src/cdk-exports.json
- Install packages for frontend React application
cd ./frontend
npm install
Testing the demo
For detailed instructions on how to use OBS to stream your local video to MediaLive, please check this link.
-
Install Open Broadcaster Software(OBS)
-
Run OBS
-
Click Settings and Streams. Choose Custom for Service.
-
Copy
MyMediaLiveChannelDestPri
from./frontend/cdk-exports.json
. The value must be in the format ofrtmp://<ip address>:<port>/<stream key>
. For Server, inputrtmp://<ip address>:<port>/
and for Stream Key, put<stream key>
. For instance, ifMyMediaLiveChannelDestPri
isrtmp://555.555.555.555:9999/streamkey
,- Server: rtmp://555.555.555.555:9999/ - Stream Key: streamkey
-
Play a video that Son Heung-min is in. It could be a local file or youtube video.
-
From Source, click
+
button and choose Window Capture. Select the window you want to start streaming to MediaLive input and click the buttonStart Streaming
-
Run the frontend application
cd ./frontend
npm start
- It will show the streaming video at the top of the page. It takes a few minutes for the backend to generate clips and analyze each clips which is 60 seconds long. Click the refresh icon at the bottom right corner, and all the clips that a certain celebrity is in will show up below.
Clean Up
- Stop Streaming from OBS by clicking button
Stop Streaming
. - Stop MediaLive Channel by clicking
Stop Channel
button.
- Since all the resources are deployed using AWS CDK, you can clean up simply by running
cdk destroy
. Although this demo uses AWS Amplify, we don’t have to runamplify delete
because no resources are created using through Amplify.
cdk destroy
Resources
License
This sample code is made available under MIT-0 license. See the LICENSE file.