Theta Infinity Studio (With Smart Contracts!)

Smart Contracts
Events

Whole project info at Theta Infinity Studio;

Inspiration

Well, this is my second entry on this hackathon, the first one was in 2021 and I didn't have much time to fully integrate with Theta Network. This is my second take on the hackathon with Theta Network fully integrated and featuring smart contracts.

Previous entry can be found here

Streaming is the future of media, no matter if you are watching tv shows, live streams, podcasts, or playing video games online.

BUT, from my perspective, we are not taking advantage of the whole potential this technology offers.

More than 10 years ago I had a job at a streaming company. I had a lot of ideas that I could never implement and Theta looks like the perfect project and technology to become motivated again.

Theta Infinity Studio is a collection of small proof of concepts that hints how multiple decentralized platforms could integrate to bring media streaming to the next level

What it does

Infinity Studio is a decentralized content management system that allows selling products through smart contracts. VoD is streamed using Theta Video API

The Project consists of two main products:

Infinity Studio

Is a Decentralized CMS for media management, with Real-time Analytics, Advertisements, eCommerce, and Real-time events management.

Infinity Player

A VideoJS wrapper provides an extra layer for extra features explained above

Features

Smart Contracts

Each product placement is associated with a smart contract and the benefits of selling goods can be split into various beneficiaries.

The smart contract is generated by Infinity itself but for now, it must be compiled using Remix and uploaded via Theta Wallet

When playing videos, previously recorded product overlays will appear and by clicking on them the transaction will be signed via Metamask

Analytics

Real-time Country Detection and Viewer Count

Real-time analytics for both videos on demand and live stream using GunJS rendered on the browser using D3.js Library

In order to get the [COUNTRY] I am relying on the service provided by DB IP In order to get the [LANGUAGE] I am relying on the information provided by the browser

VoD

stats/[PROJECT-ID]/media/views/total
stats/[PROJECT-ID]/media/views/date/[YEAR]/[MONTH]/[DATE]
stats/[PROJECT-ID]/media/views/country/[COUNTRY]/[YEAR]/[MONTH]/[DATE]
stats/[PROJECT-ID]/media/views/language/[LANGUAGE]/[YEAR]/[MONTH]/[DATE]

The order of the elements above is important because it could allow filtering by language/country, year, month, and date. This feature has not been added to the UI but the information is there to be used in the future.

VoD analytics also display the most viewed moments on the video which could be useful for better product placement.

Livestream

stats/[PROJECT-ID]/media/views/live/[CURRENT-MINUTE]/total
stats/[PROJECT-ID]/media/views/live/[CURRENT-MINUTE]/country/[COUNTRY]

[CURRENT-MINUTE] Is the number of minutes since 1970. This allows me to group values per minute, having a historic per minute of viewers and country.

Livestream analytics also display a real-time user counter at the bottom

Caveats

GunJS information is decentralized but could also be manipulated by users who would know how to access the network. This could render this feature useless when using it as a production-ready solution.

VoD Overlays

The overlays section allows users to add interactive layers above the media either served via streaming or on-demand.

VoD Overlays can trigger 3 types of overlays: Products, Advertisements, and Events.

Product Placement

Adding Dynamic Pins

In order to create a product placement, the editor must click on the record button and then track the element on the media. Pressing the mouse down will start recording the product and releasing the button will stop recording the position. Those values can be overridden by replaying the media and recording during the same time as previously recorded.

Advertisements

Ads via AdEx

Advertisements can be placed anywhere on the video and it will display an ad (320x250 is the only standard size implemented for this PoC).

Advertisements are provided by AdEx Network and its content might depend on the configuration settings on their platform. Similarly to DShop, it could be possible to implement an advertisement section inside Theta Infinity but the idea is to integrate with as many decentralized platforms as possible.

The only limitation is that AdEx requires Ad Slots to be displayed only on a single certified domain, this makes it difficult to test but nonetheless, the PoC has been successful.

Events

Events are messages sent to the player at certain moments on time. For example, you might want to change the player website background at a given point in time.

Livestream Overlays

Livestream Overlays are designed more like an extendable library than like a limited number of features. Basically, the editor can send events to all Livestream viewers.

In order to achieve this, I used Streamr Network to send real-time events that will be interpreted by the media players.

For this proof of concept I have implemented 3 real-time events:

Poll

A poll being displayed in multiple players

The poll event allows editors to notify the player that must display an interactive poll where users can vote an option (note that the voting system has not been implemented)

Rumble

Rumble is a simple event that makes the player interface shake

Animation

Animations

Real-time animation events allow editors to notify players to display given animations. These animations are JSON files provided by Lottie. I chose Lottie because it could also be used on mobile devices such as iOS or Android

In the image, you can see a Celebration Animation from Lottie> Current media players have limited options regarding advertisements or notifications, but this can be changed by adding interaction to increase the engagement

How we built it

Infinity Studio

Infinity Studio is a React-based project written in Javascript and hosted as a static website on AWS S3. Considering the platform does not use any server and all integrations are decentralized this means this tool costs almost 0 cents to maintain.

Below there is a list of the platforms this project is integrated with (at different levels)

Streamr Network: Used for Real-time events, just touching the surface of what would be possible. On this demo, I am just sending simple events but those events could not only be triggered manually by the CMS admin but also automatically or from other sources such as IoT devices.

GunJS: I could not find a decentralized analytics platform I could integrate with so I decided to implement one of my own. It's a very simple platform and it does not intend to be a professional solution.

AdEx: I used AdEx to display advertisements at certain points of the video.

IPFS: Used as a decentralized CDN to store media files and OrbitDB. For example, when creating a thumbnail strip, this is stored on IPFS.

OrbitDB: Used as a decentralized database to store information about the projects, similar to what would you do with MongoDB

Metamask: Used as a decentralized Identity provider and when signing Smart Contracts.

LottieFiles: The Infinity player also includes the Lottie library which allows the possibility of displaying any animation on top of the video, this is used when launching real-time animations on live streams.

Infinity Player

Infinity Player is written in vanilla Javascript to avoid incompatibilities with any possible future integrations (although it would normally be embedded as an iframe)

Challenges we ran into

Smart Contracts In order to have a seamless experience, I wanted to compile and publish contracts directly from the UI but I had problems with WebAssembly.

AdEx AdEx network requires the project to be hosted on a certified domain and each Ad Slot allows only one. This makes it difficult to test it locally or in different environments.

Accomplishments that we're proud of

Well, I achieved to integrate most of the platforms/technologies I thought of: Advertisements, Real-time, eCommerce, and so on

What we learned

Blockchain/ Decentralized networks are still at a very early stage and there is an important lack of platforms for dedicated markets such as Analytics, Online Bets, Gaming, IoT, ...

Some of those platforms do exist, but there is no easy integration or they are just starting