l1.media/docs/operator-guide.md

Atlas operator guide

Table of contents

• Basics
  • Introduction
  • Prerequisites
  • Running Atlas
  • Configuring Atlas
  • App name
  • App logo
  • App favicon
  • Categories
  • YouTube Partner Program
  • Terms of Service, Copyright Policy and Privacy Policy
  • Operating Atlas
  • Content blocking
  • Kill switch
• Required services
  • Orion
  • Query Node
  • RPC Node
  • Member faucet
  • Avatar service
• Optional services
  • Social previews generation
  • Price feed
  • Geolocation service
  • Captcha
• Analytics
  • Sentry
  • Livesession
  • Usersnap
  • Distribution logs

Basics

Introduction

This guide will help you get started with running your own instance of Atlas (a "gateway"). Atlas is a frontend single page application that allows users to interact with the Joystream network - basic functionalities include watching and publishing content, managing channels, issuing and trading NFTs, and more. For a brief introduction on how Atlas works and what services it interacts with, you may want to start with the architecture overview. To run a gateway, you will need to host a publicly available deployment of the frontend app, but you will also need to run a few other services that Atlas depends on. We will walk you through the process of setting up all of these services.

Tip: Atlas provides built-in "admin tools" panel that can be useful when running your own instance. You can access it in the app by pressing Ctrl+Shift+D.

Prerequisites

• Basic knowledge of how to use the command line, how to operate Docker, and how to host a web application
• Domain name (e.g. example.com) to make your gateway publicly available
• Server to deploy required services (e.g. VPS, dedicated server, or cloud instance)
• Some kind of web server (e.g. Nginx) or PaaS (e.g. Vercel) to host the frontend app

Running Atlas

To begin with, let's ignore any customization and just run pre-configured version of Atlas. As explained in the overview, Atlas repo has a monorepo structure, with multiple packages in the packages/ directory. The main package is atlas, which contains the frontend application.

Once you have cloned the repo, you can run the app in development mode by running the following commands:

yarn install
yarn atlas:dev

While the dev mode is useful for development, it is not optimized for production. To run the app in production mode, you will need to build it first:

yarn atlas:build

This will create a production build of the app in the packages/atlas/dist/ directory. You can then serve this directory using any web server, for demo purposes we will use the serve package:

yarn global add serve
serve -s -p 4200 packages/atlas/dist/

For production, you will want to use a more robust web server, such as Nginx. You can also use a PaaS such as Vercel, which will take care of building and hosting the app for you. You may find these resources useful:

• https://www.digitalocean.com/community/tutorials/how-to-deploy-a-react-application-with-nginx-on-ubuntu-20-04
• https://vercel.com/docs

Configuring Atlas

Now that you know how to run a basic version of Atlas, let's take a look at how you can make your gateway unique and tune it to your needs. The first thing you will want to take a look at is the atlas.config.yml file located in packages/atlas directory. This is the main configuration file for the app, and it will allow you to change basic setting like app name. All the configuration options have a description, so should be fairly self-explanatory. This config file also supports environment variables, so you can provide values via env like so appName: '$VITE_APP_NAME' instead of hard-coding values into the config.

Second part of the configuration is passed via environment variables. You can find a list of all the available variables in the packages/atlas/src/.env file. You can either provide those via environment or just modify the .env file. If you take a look at the file, you will notice there's a couple of different section. Sections like DEVELOPMENT_, NEXT_ and LOCAL_ are useful only for Atlas development, so if you only want to run a production instance, you should be fine ignoring those additional sections and leaving values as they are.

Note: Whenever you change the config, you will need to rebuild the app and redeploy (see Running Atlas).

App name

The most basic customization is changing the app branding. That includes app name and logo. Changing the name is as simple as changing the general.appName field in the config file. This will replace all the usage of "Atlas" in the app with your name.

App logo

Changing the logo is a bit more involved and itself consists of two steps. You will need to update the logo used in the app itself, but also the logo used in the browser favicon. To update the logo used in the app, you will need to prepare 3 versions of your logo in SVG format:

1. app-logo-short.svg - Just the logo of your app, without any text, in a square(ish) aspect ratio
2. app-logo-full.svg - The logo of your app, with the app name next to it. It should be 32 pixels high, width can vary
3. app-logo-studio.svg - Same as above, but instead of the app name, it should say "Studio" (or whatever else you desire). This is used in the Studio views.

Once you have those prepared, you should replace files with the same names located in packages/atlas/src/assets/logos/svgs/ directory. You will also need to generate React components from those SVG files. To do that, run the following command:

yarn atlas:svgs:logos

After that, all the instances of Atlas logo in the app should be replaced.

App favicon

Once you update the logo used in the app, you should also update the favicon that browsers will use in the address bar. For that purpose, you will need to generate a couple of different versions of your logo. You can take a look at the packages/atlas/src/public directory to see what assets base version of Atlas uses. Our entrypoint is the icon.svg file which also contains built-in media query for light and dark themes. There are some tools online that can help with generating the favicons set, like this one: https://realfavicongenerator.net/ although they can generate a bunch of legacy stuff that's not really needed. You can also use this article as a reference: https://evilmartians.com/chronicles/how-to-favicon-in-2021-six-files-that-fit-most-needs

Categories

Atlas uses list of categories defined in content.categories config entry to determine which content categories should be handled by the app. Categories defined in that list are local to the app - they will be used in the Discover screen and when the user is publishing new content. Each local (or "display") category also has a list of associated Query Node video categories that should be displayed inside it.

Atlas will, by default, only display content belonging to one of the defined display categories in the app. That means that video with a category that doesn't belong to any local category will not be available in the app. If you want to change that behavior, so app will display all the content regardless of the category, you can adjust this in Orion service.

You are free to use existing metaprotocol (Orion) categories, but you can also create your own that match your Gateway's specific needs. To get a list of all the existing categories you can use the following QN query:

query {
  videoCategories(limit: 200) {
    id
    name
  }
}

You can also see the current categories used by specific Gateways in Apps WG Notion: https://joystream.notion.site/5b3afe994cd64c7d8144cd2978fa5ceb?v=639feb777c514c26b2d88869d67cabf8

To create a new QN category, you can use Joystream CLI:

joystream-cli content:createVideoCategory "My category" "My category description"

Once you run the above command, you can use the query above with a orderBy: createdAt_DESC argument to get the ID of the newly created category. You can then add it to the content.categories config entry.

YouTube Partner Program

All the YPP (YouTube Partner Program) parameters could be found in atlas.config.yml file located in packages/atlas directory. features.ypp section contains all the necessary params for setting up youtube-synch and customizing user's rewards.

Once youtube-synch is setup, you'll need to provide its API URL in youtubeSyncApiUrl variable.

In order to enable YPP content in atlas, googleConsoleClientId variable needs to be provided. You can read more about it here.

You are free to use existing metaprotocol (Query Node) categories, but you can also create your own that match your Gateway's specific needs. To get a list of all the existing categories you can use the following QN query:

query {
  videoCategories(limit: 200) {
    id
    name
  }
}

You can also see the current categories used by specific Gateways in Apps WG Notion: https://joystream.notion.site/5b3afe994cd64c7d8144cd2978fa5ceb?v=639feb777c514c26b2d88869d67cabf8

To create a new QN category, you can use Joystream CLI:

joystream-cli content:createVideoCategory "My category" "My category description"

Once you run the above command, you can use the query above with a orderBy: createdAt_DESC argument to get the ID of the newly created category. You can then add it to the content.categories config entry.

YouTube Partner Program

All the YPP (YouTube Partner Program) parameters could be found in atlas.config.yml file located in packages/atlas directory. features.ypp section contains all the necessary params for setting up youtube-synch and customizing user's rewards.

Once youtube-synch is setup, you'll need to provide its API URL in youtubeSyncApiUrl variable.

In order to enable YPP content in atlas, googleConsoleClientId variable needs to be provided. You can read more about it here.

Terms of Service, Copyright Policy and Privacy Policy

You can provide your own Terms of Service, Copyright Policy and Privacy Policy by updating the legal.termsOfService, legal.copyrightPolicy and legal.privacyPolicy config entries. All of these support basic Markdown syntax like headings and lists.

Operating Atlas

Content blocking

Atlas will not display any content censored by the Joystream DAO (i.e. content marked with isCensored: true). However, you can also block additional content as the app operator. To do so, you can use blockedDataObjectIds, blockedVideoIds and blockedChannelIds config entries under content. group. Adding an ID to any of those will block respective content from being displayed in the app.

Kill switch

In case you need to take the app down for maintenance or any other reason, Atlas provides a "kill switch" mechanism that can be triggered by its operator. Enabling it will make all the users that visit the app see a "maintenance" page instead of the actual app. To enable/disable kill switch, you can use Atlas admin tools (opened with Ctrl+Shift+D in app) and navigating to "Kill switch" tab. There you can enable/disable the kill switch by providing Orion's admin secret.

Required services

Going through the previous section, your Atlas app should be configured and running. However, there are some services that the app relies on and won't be able to operate without them. In this section we will go through all of those required services.

Orion

Orion is the main backend service for Atlas - providing indexed blockchain data and sprinkling some additional information on top. It can be seen as a "Gateway node", providing everything needed for the end user and app operator. You can find more info about Orion in architecture overview. Each gateway should run its own instance of Orion, and it should be publicly available. You can find the source code for Orion in its repo. The README file in the repository provides an overview of Orion's features and basic instructions on how to run it. URL to your Orion's instance /graphql endpoint should be passed to the VITE_PRODUCTION_ORION_URL environment variable.

Query Node

Query Node (QN) is a service that processes blockchain events and stores them in a database. Orion proxies requests from Atlas to QN, so it will need a QN URL provided for it to work. You can find more info about QN in its repo. As a gateway operator you probably want to run your own QN, but it also should be fine to rely on a publicly available instance, like https://query.joystream.org/graphql.

Atlas uses Websocket connection to QN to receive real-time updates about QN state and to properly update the UI. Endpoint for this connection must be passed as the VITE_PRODUCTION_QUERY_NODE_SUBSCRIPTION_URL environment variable.

RPC Node

RPC nodes are Joystream blockchain nodes that are responsible for running the network. Whether you need your own instance of RPC node depends on your setup. If you are running your own QN, you probably also want to run RPC node alongside for speed of synching and indexing. If you make that RPC endpoint public, you can pass it as VITE_PRODUCTION_NODE_URL environment variable, meaning that all user-initiated transactions would be sent to your RPC node. However, you can also use a publicly available node, like wss://rpc.joystream.org:9944.

Member faucet

Member faucet is a service that creates free memberships for Atlas users when they sign up. You can find more information about this service in its repo. Instructions in README should be enough to configure and run your own instance. You will need to provide the URL to the faucet's /register endpoint as VITE_PRODUCTION_FAUCET_URL environment variable.

Avatar service

Avatar service is used for uploading avatars of members registered via Atlas instance. Joystream membership system is currently not connected with the storage system, meaning that only avatar URLs are accepted. To make user lives easier, Atlas will use Avatar service to upload those avatars and access them later. Code for that service is available in packages/atlas-avatar-service directory. It is also available in Dockerhub as joystream/atlas-avatar-service. You should most likely run your own instance, but it's also possible to use default one operated by Jsgenesis. Upload endpoint for that service should be passed as features.members.avatarServiceUrl config entry.

Optional services

Apart from above required services, there are a couple of optional ones that you may want to use to improve user's experience.

Social previews generation

Social previews are images that are generated when a user shares a link to a video or a channel. They are used by social media platforms to display a preview of the content. However, generating rich previews with an SPA like Atlas it not trivial. You can find more info about this issue and possible solutions in overview.

Price feed

One of optional features of the Atlas interface is showing estimated values in USD wherever JOY prices are used. To do that, you need to provide a price feed endpoint that returns a JSON with a single price field that contains the current price of JOY in USD. There is currently no service that provides such a feed, so you will need to run your own solution. The URL for that endpoint can be passed via joystream.tokenPriceFeedUrl config entry.

Geolocation service

To ensure the best possible user experience, Atlas will try to determine the user's location and use it to select content distributors physically close to the user. To do that, it uses a geolocation service that returns a JSON with latitude and longitude fields. While this is optional, it's highly recommended to enable this functionality as it will have big impact on the user experience. One of the possible solutions for this kind of service has been described in the overview. If you don't want to run your own instance, you can use Jsgenesis-operated service at https://geolocation.joystream.org. The URL for that endpoint can be passed via storage.geolocationServiceUrl config entry.

Captcha

To prevent draining the faucet, Atlas can use a captcha service to verify that the user is a human when they are creating their membership. Whether you need captcha, depends on the configuration of the faucet you are using. If the faucet requires captcha (as defined in Faucet's config), you will need to provide HCaptcha site key for Atlas to use. This should be passed as features.members.hcaptchaSiteKey environment variable. You can read more about HCaptcha in their docs.

Analytics

Atlas integrates a couple of services for product/user analytics. Those can be very helpful in understanding how your users are using the app and what they are interested in. All of those services are optional and can be disabled.

Sentry

Sentry is a service that collects errors and exceptions that happen in the app. It will keep track of what's going on in your app and notify you when something goes wrong. You can find more info about Sentry in their docs. To enable Sentry, you need to provide a DSN (identifier) which can be found in your Sentry project settings. This should be passed as analytics.sentry.dsn config entry. To disable Sentry, you can set analytics.sentry.dsn to null or remove analytics.sentry section from the config entirely.

Livesession

Livesession is a service that collects information about user sessions. It's a good way to keep track of how many users are using your app and what they are doing. You can find more info about Livesession in their docs. To enable Livesession, you need to provide an ID, which you can find in your Livesession project settings. This should be passed as analytics.livesession.id config entry. To disable Livesession, you can set analytics.livesession.id to null or remove analytics.livesession section from the config entirely.

Also, if you want to enable cross-subdomain tracking, you can provide analytics.livesession.rootHostname config entry with the root domain name. You can find more info about it here.

Usersnap

Usersnap is a service that allows users to report bugs and issues directly from the app. It's a good way to get feedback from your users and to improve your app. You can find more info about Usersnap on their website. To enable Usersnap, you need to provide an ID, which you can find in your Usersnap project settings. This should be passed as analytics.usersnap.id config entry. To disable Usersnap, you can set analytics.usersnap.id to null or remove analytics.usersnap section from the config entirely.

Distribution logs

Atlas also has a built-in mechanism for collecting end-user statistics on distributors' performance. Various response times will be collected for each asset and can then be uploaded to an assets logs service. The service should accept a JSON with a single events field that contains an array of events. Each event will contain a type field used to determine different type of events. If you want to use asset logs, you need to pass the URL of the service as analytics.assetLogs.url config entry.