semtech/mu-python-template repository overview

⁠Mu Python template

Template for mu.semte.ch⁠-microservices written in Python3. Based on the Flask⁠-framework.

⁠Quickstart

Create a Dockerfile which extends the semtech/mu-python-template-image and set a maintainer.

FROM semtech/mu-python-template
LABEL maintainer="[email protected]"

Create a web.py entrypoint-file. (naming of the entrypoint can be configured through APP_ENTRYPOINT)

@app.route("/hello")
def hello():
    return "Hello from the mu-python-template!"

Build the Docker-image for your service

docker build -t my-python-service .

Run your service

docker run -p 8080:80

You now should be able to access your service's endpoint

curl localhost:8080/hello

⁠Developing a microservice using the template

⁠Dependencies

If your service needs external libraries other than the ones already provided by the template (Flask, SPARQLWrapper and rdflib), you can specify those in a requirements.txt⁠-file. The template will take care of installing them when you build your Docker image.

⁠Development mode

By leveraging Dockers' bind-mount⁠, you can mount your application code into an existing service image. This spares you from building a new image to test each change. Just mount your services' folder to the containers' /app. On top of that, you can configure the environment variable MODE to development. That enables live-reloading of the server, so it immediately updates when you save a file.

example docker-compose parameters:

    environment:
      MODE: "development"
    volumes:
      - /home/my/code/my-python-service:/app

⁠Helper methods

The template provides the user with several helper methods. They aim to give you a step ahead for:

• logging
• JSONAPI-compliancy
• SPARQL querying

The below helpers can be imported from the helpers module. For example:

from helpers import *

Available functions:

⁠log(msg)

Works exactly the same as the logging.info⁠ method from pythons' logging module. Logs are written to the /logs directory in the docker container.
Note that the helpers module also exposes logger, which is the logger instance⁠ used by the template. The methods provided by this instance can be used for more fine-grained logging.

⁠generate_uuid()

Generate a random UUID (String).

⁠session_id_header(request)

Get the session id from the HTTP request headers.

⁠rewrite_url_header(request)

Get the rewrite URL from the HTTP request headers.

⁠validate_json_api_content_type(request)

Validate whether the Content-Type header contains the JSONAPI content-type-header. Returns a 400 otherwise.

⁠validate_resource_type(expected_type, data)

Validate whether the type specified in the JSONAPI data is equal to the expected type. Returns a 409 otherwise.

⁠error(title, status=400, **kwargs)

Returns a JSONAPI compliant error Response object⁠ with the given status code (default: 400). kwargs can be any other keys supported by JSONAPI error objects⁠.

⁠query(query)

Executes the given SPARQL select/ask/construct query.

⁠update(query)

Executes the given SPARQL update query.

The template provides one other helper module, being the escape_helpers-module. It contains functions for SPARQL query-escaping. Example import:

from escape_helpers import *

Available functions:

⁠sparql_escape ; sparql_escape_{string|uri|date|datetime|time|bool|int|float}(value)

Converts the given object to a SPARQL-safe RDF object string with the right RDF-datatype.
This functions should be used especially when inserting user-input to avoid SPARQL-injection.

Separate functions are available for different python datatypes, the sparql_escape function however can automatically select the right method to use, for following Python datatypes:

• str
• int
• float
• datetime.datetime
• datetime.date
• datetime.time
• boolean

The sparql_escape_uri-function can be used for escaping URI's.

⁠Writing SPARQL Queries

The template itself is unopinionated when it comes to constructing SPARQL-queries. However, since Python's most common string formatting methods aren't a great fit for SPARQL queries, we hereby want to provide an example on how to construct a query based on template strings⁠ while keeping things readable.

from string import Template
from helpers import query
from escape_helpers import sparql_escape_uri

my_person = "http://example.com/me"
query_template = Template("""
PREFIX mu: <http://mu.semte.ch/vocabularies/core/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>

SELECT ?name
WHERE {
    $person a foaf:Person ;
        foaf:firstName ?name .
}
""")
query_string = query_template.substitute(person=sparql_escape_uri(my_person))
query_result = query(query_string)

⁠Deployment

Example snippet for adding a service to a docker-compose stack:

my-python:
  image: my-python-service
  environment:
    LOG_LEVEL: "debug"

⁠Environment variables

• LOG_LEVEL takes the same options as defined in the Python logging⁠ module.

• MODE to specify the deployment mode. Can be development as well as production. Defaults to production

• MU_SPARQL_ENDPOINT is used to configure the SPARQL endpoint.

  • By default this is set to http://database:8890/sparql. In that case the triple store used in the backend should be linked to the microservice container as database.

• MU_APPLICATION_GRAPH specifies the graph in the triple store the microservice will work in.

  • By default this is set to http://mu.semte.ch/application. The graph name can be used in the service via settings.graph.

• MU_SPARQL_TIMEOUT is used to configure the timeout (in seconds) for SPARQL queries.

Since this template is based on the meinheld-gunicorn-docker image, all possible environment config for that image is also available for the template. See meinheld-gunicorn-docker#environment-variables⁠ for more info. The template configures WEB_CONCURRENCY in particular to 1 by default.

⁠Production

For hosting the app in a production setting, the template depends on meinheld-gunicorn-docker⁠. All environment variables⁠ used by meinheld-gunicorn can be used to configure your service as well.