Skip to content

Deploy

There are multiple options to start your own Buffy-server:

Via Docker-compose

Requirements

Server start

  • Download the buffy docker-compose file 
wget -O docker-compose.yaml https://git.connect.dzd-ev.de/dzdpythonmodules/buffy/-/raw/main/docker-compose.yaml?inline=false
  • Start the Buffy-server with docker compose
docker-compose up -d

Usage

Now your Buffy-server is available at http://localhost:8008

Via Docker image

Requirements

Start the official image

docker run -p 8008:8008 -v ${PWD}/buffy-server-cache:/data registry-gl.connect.dzd-ev.de:443/dzdpythonmodules/buffy:prod -r redis://my-redis-server

For details on the possible parameters see Via Terminal

Start your local build image

git clone ssh://git@git.connect.dzd-ev.de:22022/dzdpythonmodules/buffy.git

cd buffy

docker build . -t buffy

docker run -p 8008:8008 -v ${PWD}/buffy-server-cache:/data buffy -r redis://my-redis-server

Via Terminal

Requirements

  • Python >= 3.8
  • A running Redis Instance

Start the official pip package

pip install DZDBuffy

buffy-server

Start a local clone of the source

git clone ssh://git@git.connect.dzd-ev.de:22022/dzdpythonmodules/buffy.git

cd buffy

Optional: jump to latest stable release state 

# Get new tags from remote
git fetch --tags
# Get latest tag name
latestTag=$(git describe --tags `git rev-list --tags --max-count=1`)
# Checkout latest tag
git checkout $latestTag

Install buffy and 3rd party python modules 

pip install --no-cache-dir -e .


buffy-server

buffy-server Command Parameters

--debug

Changes python logging from INFO to DEBUG

example:

buffy-server --debug

--redis-url

Url to a running Redis database. e.g. 'redis://localhost:6379/0'.
For details on the format see https://redis.readthedocs.io/en/latest/connections.html#redis.Redis.from_url

short form: -r

example: 

buffy-server --redis-url redis://localhost:6379/0

buffy-server --r rediss://redis-server.mydomain.org:6379/0

--storage-location

Local storage location for cached requests

short form: -s

examples: 

buffy-server --storage-location /var/lib/buffy


buffy-server --s /opt/buffy

Authentication

At the moment Buffy does not support native Authentication. But you can easily put a nginx reverse proxy in front of Buffy to restrict access to Buffy.

For convenience we use a pre-configured nginx image https://hub.docker.com/r/beevelop/nginx-basic-auth / https://github.com/beevelop/docker-nginx-basic-auth

For a production environment you may want to build your own nginx setup.

On Linux install apache2-utils and create a http digest basic authentication string:

sudo apt install apache2-utils

You want to replace $with double $$ to escape them in docker-compose. a | sed "s/\\$/\$\$/g" will do this for us: 

echo "mySuperPasswort" | htpasswd -i -n MyUsername | sed "s/\\$/\$\$/g"

Lets put it all together in a docker compose file

version: "3.7"

services:
  redis:
    image: redis:alpine
    container_name: buffy_redis
    command: redis-server --save 12 1 --loglevel warning
    volumes:
      - ${PWD}/buffy-server-cache/redis:/data
  buffy-server:
    image: registry-gl.connect.dzd-ev.de:443/dzdpythonmodules/buffy:prod
    container_name: buffy_server
    volumes:
      - ${PWD}/buffy-server-cache/buffy-server:/data
  auth:
    image: beevelop/nginx-basic-auth
    container_name: buffy_auth
    ports:
      - 8008:80
    environment:
      FORWARD_PORT: 8008
      FORWARD_HOST: buffy_server
      HTPASSWD: "MyUsername:$$apr1$$oAPV7ijv$$RapWngyr74pRsF0nZY/6o1"

Authentication in the BuffyPyClient

To use our new authenticating server, we just need to supply our credentials via a requests.auth.HTTPBasicAuth instance

from buffy.buffypyclient import BuffyPyClient
from requests.auth import HTTPBasicAuth

# connect to Buffy-server with the credentials
c = BuffyPyClient(
    host="localhost",
    port=8008,
    ssl=False,
    local_download_fallback=False,
    http_auth=HTTPBasicAuth(username="MyUsername", password="mySuperPasswort"),
)

# create a request as usual
req = c.create_request(
    url="http://purl.obolibrary.org/obo/bfo.owl",
)
# save requested file
owl_file = ""
for chunk in req.download_response_content():
    owl_file += chunk.decode("utf-8")
print(owl_file)

Redis

At the Moment Redis is the only supported Backend database. This is likely too change in a future version.
Redis is easy for prototyping - thats why it is in Buffy-server atm - and great as a cache, but not best in class in being a persistent database (Redis is not ACID afaik).
To keep your Buffy-server persistent you need to start your Redis instance with the right parameters.

One solution is to create snapshots in an intervall. With docker it would look like this:

docker run -p 6379:6379 -v redis_data:/data redis redis-server --save 12 1 --loglevel warning

or via docker-compose

version: "3.7"
services:
  redis:
    image: redis
    command: redis-server --save 12 1 --loglevel warning
    volumes:
      - redis_data:/data

This would create a snapshot every 12 seconds and load the last snapshot on a reboot.

For all the details on Redis persistents with all the opitions, look into the offical docs