opa-stack

# Optional-components reference

If you havent read it, you should read about plugins first

# Configuration

TIP

You must use dynaconf merge when changing these parameters.. Example

myenv:
  OPTIONAL_COMPONENTS:
    dynaconf_merge: true
    WALRUS:
      LOAD: "auto"
      DRIVER: "redis-walrus"
      OPTS:
        URL: "redis://redis"

Take a look at the default configuration here or see the file below (it is under the OPTIONAL_COMPONENTS key) for some examples.

default-settings.yaml 
default:
  MODE: "prod"
  PROJECT_NAME: "opa-stack"
  PROJECT_DESCRIPTION: ""
  PROJECT_VERSION: "0.1.0"

  # Urls to automatic documentation. Set to null to disable
  DOCS_URL: "/docs"
  REDOC_URL: "/redoc"
  OPENAPI_URL: "/openapi.json" # Can only be null if DOCS_URL and REDOC_URL is also null

  OPENAPI_PREFIX: ""

  PLUGIN_PATHS: []

  PLUGIN_WHITELIST_LIST: []
  PLUGIN_WHITELIST_RE: ""
  PLUGIN_WHITELIST_TAGS: []
  PLUGIN_BLACKLIST_LIST: []
  PLUGIN_BLACKLIST_RE: ""
  PLUGIN_BLACKLIST_TAGS: []

  # CORS
  ALLOW_ORIGINS: ["*"]
  ALLOW_CREDENTIALS: true
  ALLOW_METHODS: ["*"]
  ALLOW_HEADERS: ["*"]

  SECRET_KEY: ""

  # Configuration for optional components..
  # They all have a helper-file in ../utils/{component_name.lower()}.py, so check
  # in them if you wonder how these values are used. They are also listed in the docs
  # @ https://opa-stack.github.io/guide/components.html
  #
  # The LOAD option is always present, and can be one of auto|yes|no, all defaulting to 'auto'.
  #   * auto: Makes opa-stack look for the component using a simple dns-check or other simple checks
  #           Ie.. Check if we "should" be able to use this component.
  #           It will then try to connect as if you had written "yes", ie, it fails if it is not able to..
  #   * yes: Makes the coponent required
  #   * no: Makes it not check at all.

  OPTIONAL_COMPONENTS:
    MONGODB:
      LOAD: "auto"
      DRIVER: "mongodb-async-motor"
      OPTS:
        URL: "mongodb://mongo:mongo@mongo:27017/opa"

    WALRUS:
      LOAD: "auto"
      DRIVER: "redis-walrus"
      OPTS:
        URL: "redis://redis"

    AIOREDIS:
      LOAD: "auto"
      DRIVER: "redis-aioredis"
      OPTS:
        URL: "redis://redis"

    CELERY:
      LOAD: "auto"
      DRIVER: "celery"
      OPTS:
        BACKEND_URL: "redis://redis/1"
        BROKER_URL: "pyamqp://guest@rabbitmq//"

  # Used different places, currently:
  #  * Setting FastAPI/Starlette debug-mode (https://www.starlette.io/applications/)
  DEBUG: false

  # Turns on better exceptions, ie, prettier, and with some more info (like variables)
  BETTER_EXCEPTIONS: false

  # Python Tools for Visual Studio debug server, running on port 5678
  PTVSD: false

  # External js files are loaded from the internet as default
  SELF_HOSTED: false

dev:
  MODE: "dev"
  PTVSD: true
  BETTER_EXCEPTIONS: true
  BETTER_EXCEPTIONS_MAX_LENGTH: 1000
  PLUGIN_PATHS:
    - "/data/opa/demo-plugins"
    - dynaconf_merge

# Usage

When you want to use an optional-component you the best way is to use opa.get_instance(name) like this

from opa import get_instance, get_router
from opa.plugins.driver_redis import Walrus

router = get_router()

@router.get("/")
def counter_sync():
    walrus = get_intance('walrus')
    counter = walrus.incr('counter')
    return f'Counter is {counter}'

WARNING

opa.get_instance() takes the lowercase component key-name as input

The other ways to get the instance (might be usefull in some cases)

  • Use component = opa.get_component(name)
    • The instance will be under component.instance, but there are other component-info you might want under component
  • Use pm = opa.get_plugin_manager(), then pm.optional_components[name] to get the component instance

# Drivers

Backend Library Drivername Async
Redis Aioredis redis-aioredis yes
Redis Walrus redis-walrus no
Mongodb Motor mongodb-async-motor yes
Celery Celery celery no

# Redis

# Aioredis

# Walrus

  • Very powerful redis library with support for a lot of additional features

# Mongodb

# Motor

# Celery / tasks

If you need basic background tasks, look into FastAPI/Starlette BackgroundTasks instead.

There is builtin support for running celery tasks, see the sample-project for details, view info about the sample project here

The opa-stack's api-container can run in worker mode, ie, it will run a celery worker with the same codebase as the rest of the api. This is by far the simplest setup, but feel free to run your own workers if you need.

  • Set environment-variable MODE=worker to start it as a worker
  • Set additional celery params (celery worker -A opa.main ${CELERY_PARAMS})

Some important notes about using celery in opa-stack is:

  • The tasks must be defined in tasks.py, ie, you will need your plugin to be a python package (folder with __init__.py).
  • You can access optional-components as usual in the tasks, but only non-async drivers are working. Ie, you can't use drivers like motor or aioredis in your tasks.
    • It is important that you DONT import any of your tasks on top of the file where your route is defined. Ie, do the from packagename.tasks import sometask inside your @router.get('/')... function!
  • The celery worker is using most of the opa-stack code. So configuration, driver-loading, hooks and so on will also work in the celery worker.

← Examples →