How OWN3D handles up to 6 Million Active Webhooks with Twitch’s new EventSub API

At the last TwitchDev Developer Day in 2020, Twitch announced a new feature, called EventSub. EventSub allows developers to subscribe to events and receive notifications over many different transports like: webhooks, websockets, etc..

We at OWN3D rely heavily on webhooks and other products from Twitch, which are mainly used for our alerts system. For example to display follows, subs, raids, hosts and other events.

Reasons for Twitch’s EventSub

Before the EventSub existed, we had to get a lot of data from various API products from Twitch, including the old Webhooks System or IRC. The bigger our product got, the more complex became the tasks for us to maintain all Twitch APIs.

At OWN3D, we currently have over half a million users with our Twitch Extensions like Free Alerts Twitch Extension, and OBS Plugin like OWN3D Pro. The following examples were a great challenge for us:

• API rate limit to renew webhooks.
• Webhooks rate limit to subscribe to webhooks.
• Scaling issues with the Twitch IRC product.
• Missing APIs to recive events like raids.

With EventSub we now have the following benefits:

• Less API overhead to maintain webhooks.
• Webhooks do not expire anymore.
• API limits are now dynamic thanks to RFC-0014.

Twitch EventSub and OWN3D ❤

In order to solve our problems in the long term we decided to develop a central microservice which is responsible for subscribing and unsubscribing and process all twitch webhooks and other events and forward them to our internal apps. We call this system the Notify Backend.

When a streamer becomes inactive, we only keep webhooks that are relevant for us to detect if the user has become active again or has disconnected his Twitch Account from OWN3D. Here is a list of EventSub topics we need to register per streamer in order for our system to operate:

• channel.update
• channel.follow
• channel.subscribe
• channel.cheer
• channel.raid
• stream.online*
• stream.offline
• channel.channel_points_custom_reward_redemption.add
• channel.channel_points_custom_reward_redemption.update
• user.authorization.revoke*

* These events are always subscribed and relevant for us to detect if a user has become active or inactive.

Creating the EventSub Libaray for our Microservice

In order to be able to use EventSub at all, we first had to modify the Twitch API client that we use for our projects. If you are interested in the exact PHP implementation of EventSub, you can find our pull request on GitHub.

On the TwitchDev Discord server you can also find other API clients for other programming languages, such as JavaScript.

Migrating to Twitch’s EventSub

Since our current infrastructure is also a microservice, the migration was very easy for us. Our new Notify Backend microservice wraps Twitch’s EventSub API so that we were very close to the Twitch standard for our internal apps.

The Notify Backend ended up doing the mapping for the Twitch EventSub and our internal apps. So we can register multiple internal apps with just one EventSub Webhook:

POST /api/v2/notifysubs/subscribe
Client-ID: 40084938-23b8-4cd8-ba76-4184ecd84981
Accept: application/json
Content-Type: application/json; charset=utf-8

{
    "types": ["follow", "sub", "re-sub", "cheer", "raid", "host"],
    "version": "1",
    "condition": {
        "platform": "twitch",
        "platform_id": "106415581"
    },
    "transports": [
        {
            "method": "webhook",
            "callback": "https://example.com/api/notifysubs/callback",
            "secret": "some-secure-secret"
        }
    ]
}

Things we learned while implementing EventSub

Back then, we built our APIs so that our microservice would send the events directly to Twitch PubSub (for the Twitch Extension) or to Pusher (for OWN3D Pro). However, this caused problems because we were not flexible when a new internal app also needed those event data.

{
  "channel_id":"106415581",
  "capabilities":["webhooks"],
  "notifies":["pusher", "twitch"],
  "broadcaster_type":"affiliate",
  "login":"ghostzero",
  "own3d_id":"1"
}

We have now changed this so that our apps can now define their own transports. We adopted this concept from Twitch’s EventSub.

Conclusion

EventSub is not based on Helix and can therefore implement new features more quickly. So we hope to see more events in the coming months. In addition, event delays have dramatically reduced. By redesigning our microservice, we now also have more flexibility when developing new internal apps that depends on event data from Twitch.

Addendum of 10/11/2021: In the meantime, we have reached 6 million. Without the EventSub we would not be able to process the amount of data from various sources.

--

We're building the future streaming tool for live entertainment, and we want your help to make it even better. Check out our career site to find out what it is like to work at StreamTV and how to join our team