All right. So, thank you for joining me on this part four on Hydra v3.
Now this is largely just going to be about what Hydra is but we are working towards v3 which is a major milestone for us in terms of the functionality that's needed,
and we really think that with this release it's really becoming possible for people to build very powerful applications,
front-end applications for substrate chains using Hydra, so we're extremely excited about it.
As I’ll get to, you know, it's actually a pretty astounding achievement to be able to build and manage this on top of everything else we're doing
because you will find many other projects that are very large teams entirely devoted to building something like Hydra,
so it's something we're really proud of, and we want to assist other people in adopting as well.
So, Hydra. What is the, I guess, the best way to understand it is just in terms of what problem is it solving.
Imagine a hypothetical blogging blockchain which is sort of a substrate chain which has the single purpose of sort of implementing some kind of a social blogging platform,
and, in fact, a one well-known substrate project called Subsocial which is covered under my little video in the top bottom right corner actually has implemented Hydra
so it's not entirely hypothetical, but just for the sake of an argument imagine this kind of a blockchain.
So, you have users submitting extrinsics or transactions where they make threads and posts, this sort of stuff.
So, that's pretty, you know, simple.
And then you can imagine building some sort of an application that's supposed to display something about this blogging infrastructure
like allowing people to post stuff, allowing people to read what's happening across different blogs and so on.
So, the naive way you would do it and the way most apps for substrate have been built is that you just build some front-end app,
you hook it up to your substrate full node, and it queries it in order to ask some simple questions about what the structure of the blog is and who's doing what and so on.
The problem that you'll pretty quickly run into is that there are a bunch of very simple queries that are needed in order to render the sort of user experience
that people are used to, certainly in Web 2.0 world that just are not possible if you ask a full node directly.
If you ask for any of these examples and really any number of other examples you could think of as they are totally reasonable, the full node won't have a pre-prepared
index over its history and state which would allow you to easily query and ask for those.
The thing that you see people doing is either you have like a front-end app which takes a very long time to sync up because it downloads everything
or a large chunk of either what's in the state or history in order to do a bunch of processing on the client side in order to show the right thing for the user.
That's slow, complicated, in general just doesn't really scale.
This is really the problem that you'll run into writing really any blockchain application, you will, if you're going to make something that's,
I would say, has a non-trivial user experience, you're going to have to somehow solve this problem.
And specifically for substrate chains, Hydra is the framework approach we've taken to solve this.
So, what it is? It's a software framework where it makes it very easy for someone who's developing a substrate chain,
such as Joystream, to focus only on the parts that are relevant to them.
They get this whole set of tools and nodes that automatically does everything else they need in order to provide this API basically which can,
for example, respond to the sort of queries that I showed you before.
So, that's the Hydra framework.
We actually were really proud to see that the Hakusama judges picked it as the first entrant winner, so that was pretty cool in the,
I believe it was the open category if I hadn't… I think that was, yeah, open hack, so that was really cool.
For some of you, maybe the way I am describing it, you may feel like you've heard of this before, in particular coming from the Ethereum space,
and this is basically because this is very very similar to what the Graph tries to do for Ethereum.
Basically, it tries to give that kind of a service for smart contracts whilst we do it for standalone chains.
There are some big important differences between the Graph and the Hydra framework.
One important difference is, well, at least before the way the Graph used to work was that the Graph company sort of hosted a service where everyone who built an app
that was talking to the Ethereum chain would sort of just talk to a server that the Graph company was running.
They were sort of not that happy with that, it's not really sort of in the spirit of the Web 3 vision
so they always had the goal of building this distributed peer-to-peer type of network that would replace their role in provisioning that service.
That's not an easy thing to do but that's something they've started to roll out, so I think over the next coming months
or so there's going to be some version of what the Graph, the hosted version of the Graph does for that is decentralized in some way.
I mean I could get into the details of what that is but I think that would be a big distraction here.
The way we run Hydra and generally people are expected to run Hydra is that the person who hosts the front-end application
would pretty much run the query node instance.
That's sort of the way we're envisioning this being provisioned.
Maybe that we end up shipping a working group which has people running query nodes which, this is what we call Hydra nodes basically,
where people are incentivized by our DAO to run them for the benefit of people using either Atlas or Pioneer or any other
major front-end application but this is one of those decisions that we are still quite early on in terms of how it's going to be provisioned at scale.
How is it that this actually works for a developer?
What a developer has to do is they have to define two things.
The whole point of Hydra is to alleviate the burden of having to do everything like talking to the chain,
and managing a database, and putting your events in there, and making an API.
It's a lot of work to get that to happen every time for a new substrate chain.
So, what a substrate developer has to do is, first, they have to just define the way the data in their system is organized
in a really nice simple sort of GraphQL like markup language.
There you would say, for example, if we take the Subsocial example that you have, you know, a blog, and you have posts, and blogs have authors, and posts are part of blogs.
You would sort of define a very convenient way, in a way that developers are very comfortable with, how the data is organized.
And then you would define what are called mappings which are basically rules which say when I see this kind of an event or this kind of a transaction
in the substrate chain, I'm going to put something in the database which then will be queryable later.
Either put something or update something or delete something, basically, update the database that holds the information that the front-end apps are interested in.
If you provide these two, you basically get everything else for free.
So, the way Hydra sort of works in production is that your application talks to a GraphQL server which has the API,
that's the API which will allow you to ask those pretty questions that I mentioned in the beginning of the slide deck here that talks to a specific database
which holds the data that I just talked about which is managed, sorted by these mappings.
Then there is a processor which is this long running process that runs these mappings whenever it sees that the underlying full node
has produced some new blocks and some new events and some new transactions.
Basically, this indexer database holds a sort of long-standing index of all the stuff that's happened in your substrate full node.
That is the basic architecture that makes a single Hydra node sort of come together.
You can basically think of the mappings as defining how the query database looks, and then you can think of the mappings as the logic that runs in the processor.
It's quite a nice abstraction.
We are extremely proud of having been able to have done that on a relatively small team.
A lot of these abstractions have been identified by the Graph, and they've done an amazing job, but it certainly has not been
easy to do this with a smaller project with a separate purpose.
We're very happy about having developed this, and we hope more people will continue to adopt it.
That's the story on Hydra of which v3 is the next major release.
So, that's it, see you for the next video.