NAME

oml2-server - OML2 server daemon

SYNOPSIS

oml2-server [-D dir | --data-dir=dir] [-H hook | --event-hook=hook]
            [-l port | --listen=port] [--user=UID] [--group=GID]
            [-t idleto | --timeout=idleto]
            [-d loglevel | --debug-level=loglevel] [--logfile=file]
            [-b db | --backend=db] [--pg-host=host] [--pg-port=port]
            [--pg-user=user] [--pg-pass=pass]
            [--pg-connect=conninfo]
            [--usage] [--version | -v] [-? | --help]
            [OML-OPTIONS]

DESCRIPTION

oml2-server (OML2 Server) is the daemon program for OML. It collects measurements from client applications and stores them in a measurement database.

oml2-server listens for connections from clients. It is normally started at boot as a system service. Each client declares which experiment it belongs to (using the --oml-exp-id option of the linkoml:liboml2 client library), and then sends a stream of measurement packets. The measurements for each client are stored in the experiment database belonging to the client’s experiment.

If the client declares that it belongs to experiment A, then oml2-server will look for an existing experiment database named A and add measurement data to that database. Otherwise it will attempt to create a new database of that name, initialize it for use, and append new measurements to it. Measurement streams from subsequent clients for experiment A will also be appended to the new database.

oml2-server can store measurements in either an SQLite3 database on disk or in a PostgreSQL database. See the --backend option for details.

Finally, runtime statistics about the server can be reported over OML, either to the server itself or, better, to another oml2-server. The available measurements points are described below, in section RUNTIME MEASUREMENT POINTS.

OPTIONS

-D directory, --data-dir=directory

Store SQLite3 measurement databases for all experiments in the specified directory, when the SQLite3 backend is selected. The default on this system is /usr/local/var/oml2. --data-dir overrides the OML_SQLITE_DIR environment variable. If oml2-server is run with an effective user id that does not have the right to create files in the directory, the server will exit with an error message in its log file. The SQLite3 database file name for an experiment is chosen by appending the suffix ".sq3" to the experiment name.

-H hook
--event-hook=hook

Specify an external hook program to call on specific events. This hook should take commands on stdin, and write reports on stdout. The hook’s stderr is the same as the oml2-server's. When first started, it should identify itself to the oml2-server by printing the following banner line.

OML HOOK READY

It should then process its input in a loop, looking for the commands described in the HOOK COMMANDS section, below. Apart from its banner, the oml2-server does not currently read anything the hook program outputs on its stdout.

A simple example hook, oml2-server-hook.sh, is available in /usr/local/share/oml2, which pushes the SQL dump of a just-finished experiment to IRODS.

-l port
--listen=port

Listen for measurement client connections on the given port. The default port is 3003.

--user=UID, --group=GID

Try to change the server’s user id and group id before starting to serve clients. OML only needs access to a single directory in which it has the right to create the SQLite3 experiment database files, plus the right to open and write to its log file (creating it if necessary). The --user and --group options allow the system administrator to run oml2-server under a non-privileged account, thereby lessening its potential as a security risk. If only --user is given, the gid will be set to the primary group id for the given user. --user and --group accept user and group names as arguments; numeric user and group ids are not supported.

-d, --debug-level=level

Set the verbosity of log output to level. The level should be an integer from 1 to 4 (1=ERROR, 2=WARNING, 3=INFO, 4=DEBUG). The default log verbosity is 3, INFO.

-t idleto
--timeout=idleto

Set the time, in seconds, after which an idle (i.e., not receiving data) client socket is reaped. This is useful to preserve system resources in the case of long running experiments, intermittent reporting or faulty reporting nodes or network. Defaults to 60s.

--logfile=file

Output log messages to file rather than stderr.

-b db, --backend=db

Select which database backend to use for storing experiment databases. The default is sqlite which stores databases as SQLite3 files. The other option is postgresql which will attempt to connect to a PostgreSQL database server.

--pg-host=host

Specify the database server to which the PostgreSQL backend should connect. Defaults to localhost.

--pg-port=port

Specify the database server port to which the PostgreSQL backend should connect. Values from /etc/services suitable for getservbyname(3) are also accepted. Defaults to postgresql (i.e., use PostgreSQL’s default, which is usually 5432).

--pg-user=user

Specify the PostgreSQL database user to use when the PostgreSQL backend is selected. The specified user must have the PostgreSQL CREATEDB role so that it can create a new database for each new experiment. Defaults to oml.

You can create this user in PostgreSQL as follows (where postgresql is the admin user for the database, and oml is the user being created).

$ createuser -U postgres --pwprompt --no-superuser --createdb --no-createrole --no-replication oml

Doing so will ask you to enter and confirm the password for this role. It should match that passed to --pg-pass below. Alternatively, the role can be created through and interactive console.

$ psql -U DBADMIN
psql (9.3.2)
Type "help" for help.

DBADMIN=# CREATE USER oml WITH PASSWORD 'oml';
CREATE ROLE
DBADMIN=# ALTER ROLE oml WITH CREATEDB;
ALTER ROLE
--pg-pass=pass

Specify the password of the pg-user to connect to the PostgreSQL backend. Defaults to the empty string.

--pg-connect=conninfo

For the PostgreSQL backend, this option gives a way to control the database authentication scheme to use. It is passed to the PostgreSQL connection API with the user name and database name appended. For instance, for an experiment named "myexp" and --pg-user=oml2, a conninfo string of "host=postgres.example.net password=secret" will result in the following being passed to the server:

host=postgres.example.net password=secret user=oml2

Though parameters which have specific --pg- options can be passed this way, unpredictable behaviours might be encountered if both --pg-X and --pg-connect specify these. It is therefore recommended to prefer the --pg-X variant, and only use --pg-connect if one doesn’t exist.

If more than one key=value pair must be passed as conninfo, the shell might make things difficult. A workaround is to wrap the entire argument in quotes, that is:

oml2-server --pg-user=oml2 "--pg-connect=host=postgres.example.net password=secret"
--logfile=file

Write log messages to file.

--usage

Print a brief usage message.

-v, --version

Print the version number of oml2-server.

-?, --help

Print a summary of options.

OML-OPTIONS

These options control the reporting of runtime metrics over OML. They are documented in liboml2(1). The avaialble MPs are documented in section RUNTIME MEASUREMENT POINTS, below.

RUNTIME MEASUREMENT POINTS

The oml2-server is able to expose some internal parameters and events over OML. While the server should be able to report these to itself, it is recommended to use a second oml2-server for the purpose of monitoring.

Currently, the following MPs are available.

clients

This measurement point records client events, and reports their address, remote port, OML node_id, experimental domain, appname, as well as the observed event (e.g., connection or disconnection), and a potential message.

An oml2-scaffold(1) application description listing these MPs can also be found in /usr/local/share/oml2.

HOOK COMMANDS

The external event hook program should expect some of the following commands on its standard input.

EXIT

Command the hook to cleanup and terminate. This command has no argument.

DBCREATED uri

New database uri (see DATABASE URI SCHEMES below) has been created ("New" here means that the database didn’t have a start_time metadatum; this is an idiosyncracy of the current code, but this essentially behaves as one would expect). Tasks such as giving permissions to specific user groups can be performed in response to this event.

DBOPENED uri

Database uri was re-opened.

DBCLOSED uri

The last client for database uri has now exited and cleanup tasks on the data can be performed.

DATABASE URI SCHEMES

Database URI schemes vary with the backend, and are as follows. * PostgreSQL: postgresql://USER@HOST:PORT/dbname where USER, HOST and PORT are the values of the equivalent --pg-* command-line options, and DBNAME is the name of the experimental domain. This scheme is PostgreSQL’s own, and is therefore directly usable with other tools.

  • SQLite3: file:fullpath where fullpath is the full path to the database in the oml2-server's local filesystem.

ENVIRONMENT VARIABLES

OML_SQLITE_DIR

Create SQLite3 experiment measurement databases in this directory. This environment variable is overridden by the --data-dir command line option.

SIGNALS

The oml2-server reacts to some signals.

SIGUSR

Log an information message reporting the status of its allocated memory

SIGINT & SIGTERM

Gracefully terminate, emptying the buffers and closing client connections.

LOAD-BALANCING SERVERS

For heavy measurement load scenarios, it is possible to load-balance client streams to multiple instances of the oml2-server. This adds fault-tolerance and scalability to an existing OML server/clients setup. The approach relies on the existing TCP load balancer, such as HAProxy.

The OML client’s collection URL needs to be pointed to the machine and port on which the haproxy(1) listens. The proxy will in turns redirect the stream to one of the available oml2-server instances.

Load-balancing in haproxy(1) can be enabled as follows.

listen oml2server :3003
        mode tcp
        option tcplog
        balance leastconn

        server oml2server1 192.0.2.1:3003 check
        server oml2server2 192.0.2.2:3003 check
        server oml2server3 192.0.2.3:3003 check

HAProxy supports different decision algorithms to load-balance incoming TCP flows. This example uses the leastconn algorithm, which selects the server with the least number of connections. A full example configuration file can be found as /usr/local/share/oml2/haproxy.conf.

Each server can store data independently into the same or separate (and potentially different) backends. In the former case, the database server can be separately load-balanced (e.g., PostgreSQL clustering). This is however out of the scope of this documentation.

SECURITY CONSIDERATIONS

oml2-server does not use any authentication, and should thus be considered insecure. It is intended to be deployed behind firewalls on a dedicated testbed network. It should not be run as a daemon on an open network. Future versions of OML may be re-designed to be suitable for use in insecure environments.

Currently, the only way to pass credentials for the PostgreSQL backend is via the command line --pg-pass or --pg-conninfo options. To avoid making such sensitive information available to the casual ps(1) user, the oml2-server attempts to overwrite these parts of its command line as soon as feasible, overwriting the entire argument. This is a poor man’s security measure, as this creates a race condition where, prior to doing this, the credentials are still visible to everybody. Moreover, after replacement, the length of the password is still obvious when using --pg-pass.

BUGS

If a problem you are experiencing is not addressed in the FAQ (http://oml.mytestbed.net/projects/oml/wiki/FAQ_and_Support) nor already present in the list of know bugs (http://oml.mytestbed.net/projects/oml/issues). You could discuss it on the mailing list (details and archives at http://oml.mytestbed.net/tab/show?id=oml).

It is however advisable to open a ticket on our issue tracker at http://oml.mytestbed.net/projects/oml/issues/new. Don’t forget to include details such as client and server logs (at [--oml-log-level|-d] 2). It also helps if you can share the source code of a (minimal, if possible) example reliably triggering the problem.