Joystream Stats 97777b0621 WIP use joystream-lib пре 2 година
..
img 2c9a662c65 more README: joystream-lib + banner пре 2 година
joystream-api 238bf9008f move contributions/tech to scripts пре 2 година
joystream-leaderboard-bot 238bf9008f move contributions/tech to scripts пре 2 година
joystream-scoring-period-bot 238bf9008f move contributions/tech to scripts пре 2 година
joystreamtelegrambot 97777b0621 WIP use joystream-lib пре 2 година
joystreamvideobot 238bf9008f move contributions/tech to scripts пре 2 година
minting-burning-report 63d42cc2ef move `council/joystream-api` to `minting-burning-report` пре 2 година
node-setup-script 238bf9008f move contributions/tech to scripts пре 2 година
report-generator 238bf9008f move contributions/tech to scripts пре 2 година
sp-downloader-tester 238bf9008f move contributions/tech to scripts пре 2 година
storagesize-bot 238bf9008f move contributions/tech to scripts пре 2 година
substrate_polkadot_content_list 238bf9008f move contributions/tech to scripts пре 2 година
validator-report-backend 238bf9008f move contributions/tech to scripts пре 2 година
validator-report-ui 238bf9008f move contributions/tech to scripts пре 2 година
README.md 2c9a662c65 more README: joystream-lib + banner пре 2 година

README.md

Community Scripts

Here you find scripts and info how to retrieve and mutate data on joystream blockchains.

There are several ways to access the chain storage. The easiest is the governance app pioneer.

Scripts

Webapps

Telgram & Discord Bots

Contact

For usage questions: #tech-support. To join development come to #operations and check Operations reports.

Chain State

First select Fully Featured for interface operation mode on Settings and click Save.

Go to Chain State and select section and method. Generally it is possible to disable the input field to retrieve a list of all entries (equivalent to for example api.query.system.account.entries() - see double maps).

To see the balance of an account navigate to system.account, select or enter a key and press the + button.

When you find a member ID and want to know, who it is, use members.membershipById.

If the handle is known: members.membershipById. The result is the membership ID to retrieve the membership like above. It expects hexadecimal bytes prefixed with 0x as input:

  • Use a hex converter and prefix it with 0x.
  • With node.js: Buffer.from("handle").toString('hex'))
  • On JS enter console.log('handle'.split("").map(c => c.charCodeAt(0).toString(16).padStart(2, "0")).join("")) (save as string to hex for later by clicking on the disk button)

With the member info two keys are included, root_account and contoller_account. These are usually equal, except someone changed the controller.

members.membershipById: Membership
{
  handle: joystreamstats,
  avatar_uri: https://joystreamstats.live/favicon.ico,
  about: https://joystreamstats.live,
  registered_at_block: 1,
  registered_at_time: 1,613,791,830,000,
  entry: {
    Genesis: null
  },
  suspended: false,
  subscription: null,
  root_account: 5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3,
  controller_account: 5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3
}

You can try this key with system.account to get the current balance.

With the key more lookups are possible:

JS

To use the Javascript page in Pioneer first select Fully Featured for interface operation mode on Settings and click Save.

Open Toolbox), select one of the pre-defined code snippets and press the play button to execute it.

You can edit the code or paste your own script. If after editing the code nothing happens the browser console (Ctrl + Shift + I) can help to debug the problem.

For above example to get a membership enter:

const handle = 'joystreamstats'
const hex = handle.split("").map(c => c.charCodeAt(0).toString(16).padStart(2, "0")).join("")
const memberId = await api.query.members.memberIdByHandle(`0x` + hex)
const member = await api.query.members.membershipById(memberId)
console.log(handle, memberId)

// nicely print the member object
Object.keys(member).map(key=> console.log(`${key}: ${JSON.stringify(member[key])}`))

Result:

joystreamstats 1305
registry: {}
createdAtHash: undefined
handle: "joystreamstats"
avatar_uri: "https://joystreamstats.live/favicon.ico"
about: "https://joystreamstats.live"
registered_at_block: 1
registered_at_time: 1613791830000
entry: {"genesis":null}
suspended: false
subscription: null
root_account: "5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3"
controller_account: "5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3"
typeDefs: {}

Useful functions:

  • .hash(): Displays the storage hash
  • .size(): Display the storage entry size
  • .at(): Display the historic value at a given block hash (see below)
  • .key: Display the key for a double map
  • .keyPrefix: Display the key prefix for a double map

Subscribe to balance changes for multiple accounts:

api.query.system.account.multi([ALICE, BOB], (info) => {
  console.log('Change detected, new balances: ', info)
});

(See example Listen to multiple balances changes)

Show events between two blocks:

const start = 2000200
const end = 2000500
const hide = [ 'ExtrinsicSuccess', 'HeartbeatReceived' ]

console.log(`--- Events between ${start} and ${end} ---`)
for (let block = start ; block <= end ; ++block) {
 const hash = await api.rpc.chain.getBlockHash(block)
 api.query.system.events.at(hash).then((events) => {
  // filter hidden event methods
  const filtered = events.filter(({event}) => !hide.includes(event.method))
  if (!filtered.length) return
  console.log(`#[${block}: ${events.length} event(s)`);
  // loop through the Vec<EventRecord>
  filtered.forEach((record) => {
  // extract the phase, event and the event types
    const { event, phase } = record;
    const types = event.typeDef;
    // show what we are busy with
    console.log(` - ${event.section}.${event.method}::${phase}=${phase.toString()}`);
    console.log(event.meta.documentation.toString().padStart(`\t`));
    // loop through each of the parameters, displaying the type and data
    event.data.forEach((data, index) => {
      console.log(types[index].type + ';' + data.toString());
    })
  })
 })
}
console.log(`---`)

See augment-api-events.ts for a list of all extrinsics. Note that this runs much faster with a script connected to a local node.

joystream-cli

Another useful even tough at first time consuming tool to inspect working groups for example is the cli:

git clone https://github.com/Joystream/joystream
cd joystream/cli
yarn
yarn link || bash # Start a new shell after linking `cli`
./bin/run api:setUri # Select the default api endpoint
./bin/run working-groups:overview
./bin/run working-groups:openings

To select another group: -g storageProviders, curators, operations[Alpha,Beta,Gamma]

api

All methods in Chain State are available using the api

To start try this compact example in joystream-api/src/general/status.ts to query the chain:

git clone https://github.com/Joystream/community-repo
cd community-repo/scripts/joystream-api
community-repo/scripts/joystream-api$ yarn
yarn install v1.22.17
[1/4] Resolving packages...
success Already up-to-date.
$ yarn build
yarn run v1.22.17
$ tsc --build tsconfig.json
Done in 5.55s.
Done in 5.76s.
/community-repo/scripts/joystream-api$ yarn status
yarn run v1.22.17
$ node lib/general/status
Chain 'Joystream' - node: Joystream Node v5.9.0-c7711424ef-x86_64-linux-gnu
Runtime Version: 9.7.0
Council size: 20
Validator count: 69
Total Validator Locked Balances: 171694540
Done in 8.75s.

If you have no local validator node, change the provider URL in line 13, for example wss://testnet-rpc-3-uk.joystream.org or wss://joystreamstats.live:9945 (alternatively click on the blue circle in Pioneer to see more, custom endpoint shows the address). Or set one up with node-setup-script :)

The folder has more examples and it is worth to look around. You can also try queries from the scripts on the JS page directly and log the output:

  const [chain, nodeName, nodeVersion] = await Promise.all([
    api.rpc.system.chain(),
    api.rpc.system.name(),
    api.rpc.system.version()
  ]);

  console.log(`Chain '${chain}' - node: ${nodeName} v${nodeVersion}`);

For the current testnet augment-api-query.ts gives a list of all api queries with descriptions (also in node_modules/@joystream/types/augment):

/**
       * Registered unique handles and their mapping to their owner
       **/
      memberIdByHandle: AugmentedQuery<ApiType, (arg: Bytes | string | Uint8Array) => Observable<MemberId>, [Bytes]>;
      /**
       * Mapping of a controller account id to vector of member ids it controls
       **/
      memberIdsByControllerAccountId: AugmentedQuery<ApiType, (arg: AccountId | string | Uint8Array) => Observable<Vec<MemberId>>, [AccountId]>;
      /**
       * Mapping of a root account id to vector of member ids it controls.
       **/
      memberIdsByRootAccountId: AugmentedQuery<ApiType, (arg: AccountId | string | Uint8Array) => Observable<Vec<MemberId>>, [AccountId]>;
      /**
       * Mapping of member's id to their membership profile
       **/
      membershipById: AugmentedQuery<ApiType, (arg: MemberId | AnyNumber | Uint8Array) => Observable<Membership>, [MemberId]>;

joystream-lib

The joystream-lib is a collection of functions used by for example the report-generator, minting-burning-report and js:stats. It makes the code a little more readable and requires less work on type upgrades.

from generator.ts:

import { ApiPromise } from "@polkadot/api";
import { connectApi, getHead, getCouncils, getCouncilRound } from "./lib/api";
import { Round } from "./lib/types";

const providerUrl = "ws://127.0.0.1:9944"

async function printRounds () {
  const api: ApiPromise = await connectApi(providerUrl);
  await api.isReady;

  const head = await getHead(api);
  console.log(`current head: #${head}, round: ${await getCouncilRound(api)}`)
  getCouncils(api, +head).then(async (councils: Round[]) => {
    api.disconnect();
    councils.map(({ round, start, end }) => console.log(round, start, end))
    process.exit();
  });
}
printRounds()

Try to create this file in report-generator/src/rounds.ts, add "rounds": "node build/rounds", to the scripts in packages.json (after storage, note that in json the last element has no ,) and run:

cd community-repo/scripts/report-generator
yarn && yarn build
yarn rounds

Result:

yarn run v1.22.17
$ node build/rounds
current head: #4041100, round: 42
2 118800 277199
4 277200 377999
5 378000 478799
6 478800 579599
7 579600 680399
...

To use it in your project just add it as submodule like here do

cd your-project
git submodule add https://git.joystreamstats.live/Operations/joystream-lib lib
git status # should show: 	new file:   .gitmodules
git commit -m "add joystream-lib submodule"
git push

When cloning:

git clone URL project
cd project
git submodule update --init --recursive