Type something to search...
Grist

Grist

Grist

6.3k 265
01 May, 2024
  TypeScript

What is Grist ?

Grist is a modern relational spreadsheet. It combines the flexibility of a spreadsheet with the robustness of a database.


Grist Features

Grist is a hybrid database/spreadsheet, meaning that:

  • Columns work like they do in databases: they are named, and they hold one kind of data.
  • Columns can be filled by formula, spreadsheet-style, with automatic updates when referenced cells change.

This difference can confuse people coming directly from Excel or Google Sheets. Give it a chance! There’s also a Grist for Spreadsheet Users article to help get you oriented. If you’re coming from Airtable, you’ll find the model familiar (and there’s also our Grist vs Airtable article for a direct comparison).

Here are some specific feature highlights of Grist:

  • Python formulas.
  • A portable, self-contained format.
    • Based on SQLite, the most widely deployed database engine.
    • Any tool that can read SQLite can read numeric and text data from a Grist file.
    • Enables backups that you can confidently restore in full.
    • Great for moving between different hosts.
  • Can be displayed on a static website with grist-static – no special server needed.
  • A self-contained desktop app for viewing and editing locally: grist-electron.
  • Convenient editing and formatting features.
    • Choices and choice lists, for adding colorful tags to records.
    • References and reference lists, for cross-referencing records in other tables.
    • Attachments, to include media or document files in records.
    • Dates and times, toggles, and special numerics such as currency all have specialized editors and formatting options.
    • Conditional Formatting, letting you control the style of cells with formulas to draw attention to important information.
  • Drag-and-drop dashboards.
    • Charts, card views and a calendar widget for visualization.
    • Summary tables for summing and counting across groups.
    • Widget linking streamlines filtering and editing data. Grist has a unique approach to visualization, where you can lay out and link distinct widgets to show together, without cramming mixed material into a table.
    • Filter bar for quick slicing and dicing.
  • Incremental imports.
    • Import a CSV of the last three months activity from your bank…
    • …and import new activity a month later without fuss or duplication.
  • Integrations.
  • Many templates to get you started, from investment research to organizing treasure hunts.
  • Access control options.
  • Self-maintainable.
    • Useful for intranet operation and specific compliance requirements.
  • Sandboxing options for untrusted documents.
    • On Linux or with Docker, you can enable gVisor sandboxing at the individual document level.
    • On macOS, you can use native sandboxing.
    • On any OS, including Windows, you can use a wasm-based sandbox.
  • Translated to many languages.
  • F1 key brings up some quick help. This used to go without saying, but in general Grist has good keyboard support.
  • We post progress on 𝕏 or Twitter or whatever and publish monthly newsletters.

If you are curious about where Grist is heading, see our roadmap, drop a question in our forum, or browse our extensive documentation.


Using Grist

If you just want a quick demo of Grist:

To get grist-core running on your computer with Docker, do:

Terminal window
docker pull gristlabs/grist
docker run -p 8484:8484 -it gristlabs/grist

Then visit http://localhost:8484 in your browser. You’ll be able to create, edit, import, and export documents. To preserve your work across docker runs, share a directory as /persist:

Terminal window
docker run -p 8484:8484 -v $PWD/persist:/persist -it gristlabs/grist

Get templates at templates.getgrist.com for payroll, inventory management, invoicing, D&D encounter tracking, and a lot more, or use any document you’ve created on docs.getgrist.com.

If you need to change the port Grist runs on, set a PORT variable, don’t just change the port mapping:

docker run --env PORT=9999 -p 9999:9999 -v $PWD/persist:/persist -it gristlabs/grist

To enable gVisor sandboxing, set --env GRIST_SANDBOX_FLAVOR=gvisor. This should work with default docker settings, but may not work in all environments.

You can find a lot more about configuring Grist, setting up authentication, and running it on a public server in our Self-Managed Grist handbook.


Building from source

To build Grist from source, follow these steps:

yarn install
yarn run build:prod
yarn run install:python
yarn start
# Grist will be available at http://localhost:8484/

Grist formulas in documents will be run using Python executed directly on your machine. You can configure sandboxing using a GRIST_SANDBOX_FLAVOR environment variable.

  • On macOS, export GRIST_SANDBOX_FLAVOR=macSandboxExec uses the native sandbox-exec command for sandboxing.
  • On Linux with gVisor’s runsc installed, export GRIST_SANDBOX_FLAVOR=gvisor is an option.
  • On any OS including Windows, export GRIST_SANDBOX_FLAVOR=pyodide is available.

These sandboxing methods have been written for our own use at Grist Labs and may need tweaking to work in your own environment - pull requests very welcome here!


Logins

Like git, Grist has features to track document revision history. So for full operation, Grist expects to know who the user modifying a document is. Until it does, it operates in a limited anonymous mode. To get you going, the docker image is configured so that when you click on the “sign in” button Grist will attribute your work to [email protected]. Change this by setting GRIST_DEFAULT_EMAIL:

docker run --env GRIST_DEFAULT_EMAIL=my@email -p 8484:8484 -v $PWD/persist:/persist -it gristlabs/grist

You can change your name in Profile Settings in the User Menu.

For multi-user operation, or if you wish to access Grist across the public internet, you’ll want to connect it to your own Single Sign-On service. There are a lot of ways to do this, including SAML and forward authentication. Grist has been tested with Authentik, Auth0, and Google/Microsoft sign-ins via Dex.


Environment variables

Grist can be configured in many ways. Here are the main environment variables it is sensitive to:

VariablePurpose
ALLOWED_WEBHOOK_DOMAINScomma-separated list of permitted domains to use in webhooks (e.g. webhook.site,zapier.com). You can set this to * to allow all domains, but if doing so, we recommend using a carefully locked-down proxy (see GRIST_HTTPS_PROXY) if you do not entirely trust users. Otherwise services on your internal network may become vulnerable to manipulation.
APP_DOC_URLdoc worker url, set when starting an individual doc worker (other servers will find doc worker urls via redis)
APP_DOC_INTERNAL_URLlike APP_DOC_URL but used by the home server to reach the server using an internal domain name resolution (like in a docker environment). Defaults to APP_DOC_URL
APP_HOME_URLurl prefix for home api (home and doc servers need this)
APP_STATIC_URLurl prefix for static resources
APP_STATIC_INCLUDE_CUSTOM_CSSset to “true” to include custom.css (from APP_STATIC_URL) in static pages
APP_UNTRUSTED_URLURL at which to serve/expect plugin content.
GRIST_ADAPT_DOMAINset to “true” to support multiple base domains (careful, host header should be trustworthy)
GRIST_ALLOWED_HOSTScomma-separated list of permitted domains origin for requests (e.g. my.site,another.com)
GRIST_APP_ROOTdirectory containing Grist sandbox and assets (specifically the sandbox and static subdirectories).
GRIST_BACKUP_DELAY_SECSwait this long after a doc change before making a backup
GRIST_DATA_DIRdirectory in which to store document caches.
GRIST_DEFAULT_EMAILif set, login as this user if no other credentials presented
GRIST_DEFAULT_PRODUCTif set, this controls enabled features and limits of new sites. See names of PRODUCTS in Product.ts.
GRIST_DEFAULT_LOCALELocale to use as fallback when Grist cannot honour the browser locale.
GRIST_DOMAINin hosted Grist, Grist is served from subdomains of this domain. Defaults to “getgrist.com”.
GRIST_EXPERIMENTAL_PLUGINSenables experimental plugins
GRIST_ENABLE_REQUEST_FUNCTIONenables the REQUEST function. This function performs HTTP requests in a similar way to requests.request. This function presents a significant security risk, since it can let users call internal endpoints when Grist is available publicly. This function can also cause performance issues. Unset by default.
GRIST_HIDE_UI_ELEMENTScomma-separated list of UI features to disable. Allowed names of parts: helpCenter,billing,templates,createSite,multiSite,multiAccounts,sendToDrive,tutorials. If a part also exists in GRIST_UI_FEATURES, it will still be disabled.
GRIST_HOME_INCLUDE_STATICif set, home server also serves static resources
GRIST_HOSThostname to use when listening on a port.
GRIST_HTTPS_PROXYif set, use this proxy for webhook payload delivery.
GRIST_ID_PREFIXfor subdomains of form o-, expect or produce o-${GRIST_ID_PREFIX}.
GRIST_IGNORE_SESSIONif set, Grist will not use a session for authentication.
GRIST_INST_DIRpath to Grist instance configuration files, for Grist server.
GRIST_LIST_PUBLIC_SITESif set to true, sites shared with the public will be listed for anonymous users. Defaults to false.
GRIST_MANAGED_WORKERSif set, Grist can assume that if a url targeted at a doc worker returns a 404, that worker is gone
GRIST_MAX_UPLOAD_ATTACHMENT_MBmax allowed size for attachments (0 or empty for unlimited).
GRIST_MAX_UPLOAD_IMPORT_MBmax allowed size for imports (except .grist files) (0 or empty for unlimited).
GRIST_OFFER_ALL_LANGUAGESif set, all translated langauages are offered to the user (by default, only languages with a special ‘good enough’ key set are offered to user).
GRIST_ORG_IN_PATHif true, encode org in path rather than domain
GRIST_PAGE_TITLE_SUFFIXa string to append to the end of the <title> in HTML documents. Defaults to " - Grist". Set to _blank for no suffix at all.
GRIST_PROXY_AUTH_HEADERDeprecated, and interpreted as a synonym for GRIST_FORWARD_AUTH_HEADER.
GRIST_ROUTER_URLoptional url for an api that allows servers to be (un)registered with a load balancer
GRIST_SERVE_SAME_ORIGINset to “true” to access home server and doc workers on the same protocol-host-port as the top-level page, same as for custom domains (careful, host header should be trustworthy)
GRIST_SERVERSthe types of server to setup. Comma separated values which may contain “home”, “docs”, static” and/or “app”. Defaults to “home,docs,static”.
GRIST_SESSION_COOKIEif set, overrides the name of Grist’s cookie
GRIST_SESSION_DOMAINif set, associates the cookie with the given domain - otherwise defaults to GRIST_DOMAIN
GRIST_SESSION_SECRETa key used to encode sessions
GRIST_SKIP_BUNDLED_WIDGETSif set, Grist will ignore any bundled widgets included via NPM packages.
GRIST_ANON_PLAYGROUNDWhen set to ‘false’ deny anonymous users access to the home page
GRIST_FORCE_LOGINMuch like GRIST_ANON_PLAYGROUND but don’t support anonymous access at all (features like sharing docs publicly requires authentication)
GRIST_SINGLE_ORGset to an org “domain” to pin client to that org
GRIST_TEMPLATE_ORGset to an org “domain” to show public docs from that org
GRIST_HELP_CENTERset the help center link ref
GRIST_SUPPORT_ANONif set to ‘true’, show UI for anonymous access (not shown by default)
GRIST_SUPPORT_EMAILif set, give a user with the specified email support powers. The main extra power is the ability to share sites, workspaces, and docs with all users in a listed way.
GRIST_TELEMETRY_LEVELthe telemetry level. Can be set to: off (default), limited, or full.
GRIST_THROTTLE_CPUif set, CPU throttling is enabled
GRIST_TRUST_PLUGINSif set, plugins are expect to be served from the same host as the rest of the Grist app, rather than from a distinct host. Ordinarily, plugins are served from a distinct host so that the cookies used by the Grist app are not automatically available to them. Enable this only if you understand the security implications.
GRIST_USER_ROOTan extra path to look for plugins in - Grist will scan for plugins in $GRIST_USER_ROOT/plugins.
GRIST_UI_FEATUREScomma-separated list of UI features to enable. Allowed names of parts: helpCenter,billing,templates,createSite,multiSite,multiAccounts,sendToDrive,tutorials. If a part also exists in GRIST_HIDE_UI_ELEMENTS, it won’t be enabled.
GRIST_UNTRUSTED_PORTif set, plugins will be served from the given port. This is an alternative to setting APP_UNTRUSTED_URL.
GRIST_WIDGET_LIST_URLa url pointing to a widget manifest, by default https://github.com/gristlabs/grist-widget/releases/download/latest/manifest.json is used
COOKIE_MAX_AGEsession cookie max age, defaults to 90 days; can be set to “none” to make it a session cookie
HOME_PORTport number to listen on for REST API server; if set to “share”, add API endpoints to regular grist port.
PORTport number to listen on for Grist server
REDIS_URLoptional redis server for browser sessions and db query caching
GRIST_SNAPSHOT_TIME_CAPoptional. Define the caps for tracking buckets. Usage: {“hour”: 25, “day”: 32, “isoWeek”: 12, “month”: 96, “year”: 1000}
GRIST_SNAPSHOT_KEEPoptional. Number of recent snapshots to retain unconditionally for a document, regardless of when they were made
GRIST_PROMCLIENT_PORToptional. If set, serve the Prometheus metrics on the specified port number. ⚠️ Be sure to use a port which is not publicly exposed ⚠️.
VariablePurpose
ASSISTANT_API_KEYoptional. An API key to pass when making requests to an external AI conversational endpoint.
ASSISTANT_CHAT_COMPLETION_ENDPOINToptional. A chat-completion style endpoint to call. Not needed if OpenAI is being used.
ASSISTANT_MODELoptional. If set, this string is passed along in calls to the AI conversational endpoint.
ASSISTANT_LONGER_CONTEXT_MODELoptional. If set, requests that fail because of a context length limitation will be retried with this model set.
OPENAI_API_KEYoptional. Synonym for ASSISTANT_API_KEY that assumes an OpenAI endpoint is being used. Sign up for an account on OpenAI and then generate a secret key here.

At the time of writing, the AI Assistant is known to function against OpenAI chat completion endpoints for gpt-3.5-turbo and gpt-4. It can also function against the chat completion endpoint provided by llama-cpp-python.

VariablePurpose
GRIST_SANDBOX_FLAVORcan be pynbox, unsandboxed, docker, or macSandboxExec. If set, forces Grist to use the specified kind of sandbox.
GRIST_SANDBOXa program or image name to run as the sandbox. See NSandbox.ts for nerdy details.
PYTHON_VERSIONcan be 2 or 3. If set, documents without an engine setting are assumed to use the specified version of python. Not all sandboxes support all versions.
PYTHON_VERSION_ON_CREATIONcan be 2 or 3. If set, newly created documents have an engine setting set to python2 or python3. Not all sandboxes support all versions.

Forward authentication variables:

VariablePurpose
GRIST_FORWARD_AUTH_HEADERif set, trust the specified header (e.g. “x-forwarded-user”) to contain authorized user emails, and enable “forward auth” logins.
GRIST_FORWARD_AUTH_LOGIN_PATHif GRIST_FORWARD_AUTH_HEADER is set, Grist will listen at this path for logins. Defaults to /auth/login.
GRIST_FORWARD_AUTH_LOGOUT_PATHif GRIST_FORWARD_AUTH_HEADER is set, Grist will forward to this path when user logs out.

Forward authentication supports two modes, distinguished by GRIST_IGNORE_SESSION:

  1. With sessions, and forward-auth on login endpoints.

    For example, using traefik reverse proxy with traefik-forward-auth middleware:

    • GRIST_IGNORE_SESSION: do NOT set, or set to a falsy value.
    • Make sure your reverse proxy applies the forward auth middleware to GRIST_FORWARD_AUTH_LOGIN_PATH and GRIST_FORWARD_AUTH_LOGOUT_PATH.
    • If you want to allow anonymous access in some cases, make sure all other paths are free of the forward auth middleware. Grist will trigger it as needed by redirecting to GRIST_FORWARD_AUTH_LOGIN_PATH. Once the user is logged in, Grist will use sessions to identify the user until logout.
  2. With no sessions, and forward-auth on all endpoints.

    For example, using HTTP Basic Auth and server configuration that sets the header (specified in GRIST_FORWARD_AUTH_HEADER) to the logged-in user.

  • GRIST_IGNORE_SESSION: set to true. Grist sessions will not be used.
  • Make sure your reverse proxy sets the header you specified for all requests that may need login information. It is imperative that this header cannot be spoofed by the user, since Grist will trust whatever is in it.

When using forward authentication, you may wish to also set the following variables:

  • GRIST_FORCE_LOGIN=true to disable anonymous access.

Plugins:

Grist has a plugin system, used internally. One useful thing you can do with it is include custom widgets in a build of Grist. Custom widgets are usually made available just by setting GRIST_WIDGET_LIST_URL, but that has the downside of being an external dependency, which can be awkward for offline use or for archiving. Plugins offer an alternative.

To “bundle” custom widgets as a plugin:

  • Add a subdirectory of plugins, e.g. plugins/my-widgets. Alternatively, you can set the GRIST_USER_ROOT environment variable to any path you want, and then create plugins/my-widgets within that.
  • Add a manifest.yml file in that subdirectory that looks like this:
name: My Widgets
components:
widgets: widgets.json
  • The widgets.json file should be in the format produced by the grist-widget repository, and should be placed in the same directory as manifest.yml. Any material in plugins/my-widgets will be served by Grist, and relative URLs can be used in widgets.json.
  • Once all files are in place, restart Grist. Your widgets should now be available in the custom widgets dropdown, along with any others from GRIST_WIDGET_LIST_URL.
  • If you like, you can add multiple plugin subdirectories, with multiple sets of widgets, and they’ll all be made available.

Google Drive integrations:

VariablePurpose
GOOGLE_CLIENT_IDset to the Google Client Id to be used with Google API client
GOOGLE_CLIENT_SECRETset to the Google Client Secret to be used with Google API client
GOOGLE_API_KEYset to the Google API Key to be used with Google API client (accessing public files)
GOOGLE_DRIVE_SCOPEset to the scope requested for Google Drive integration (defaults to drive.file)

Database variables:

VariablePurpose
TYPEORM_DATABASEdatabase filename for sqlite or database name for other db types
TYPEORM_HOSThost for db
TYPEORM_LOGGINGset to ‘true’ to see all sql queries
TYPEORM_PASSWORDpassword to use
TYPEORM_PORTport number for db if not the default for that db type
TYPEORM_TYPEset to ‘sqlite’ or ‘postgres’
TYPEORM_USERNAMEusername to connect as
TYPEORM_EXTRAany other properties to pass to TypeORM in JSON format

Testing:

VariablePurpose
GRIST_TESTING_SOCKETa socket used for out-of-channel communication during tests only.
GRIST_TEST_HTTPS_OFFSETif set, adds https ports at the specified offset. This is useful in testing.
GRIST_TEST_SSL_CERTif set, contains filename of SSL certificate.
GRIST_TEST_SSL_KEYif set, contains filename of SSL private key.
GRIST_TEST_LOGINallow fake unauthenticated test logins (suitable for dev environment only).
GRIST_TEST_ROUTERif set, then the home server will serve a mock version of router api at /test/router
GREP_TESTSpattern for selecting specific tests to run (e.g. env GREP_TESTS=ActionLog yarn test).