Changes between Initial Version and Version 1 of General/2aTutorials


Ignore:
Timestamp:
Feb 18, 2019, 10:33:12 PM (3 years ago)
Author:
seskar
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • General/2aTutorials

    v1 v1  
     1= Quick Start Tutorial =
     2
     3[[TOC(heading=General Documentation, General/*, depth=1)]]
     4
     5The easiest way to instrument or create an OML application is by using oml2-scaffold. Starting from a description of the application's measurement points it can rapidly generate the skeleton of the OML application. The aim of this tool is to facilitate the generation of type-safe C code. Previous versions of this tutorial make use of the low-level C API functions and data structures. Although this approach is still valid, its use is discouraged because the low-level API lacks type-safety. For new applications the use of oml2-scaffold is preferred. oml2-scaffold is a standard part of the OML release. Under Debian, for example, this tool is available in the liboml2-dev package.
     6
     7== Using OML scaffold to Create the Skeleton of the Application ==
     8In this guide we will use the OML scaffold to re-create the "sine generator" example application which is shipped with the OML source (source:example/liboml2). This simple example demonstrates both an OML-instrumented application as well as the code-generating capabilities of oml2-scaffold. oml2-scaffold itself is a Ruby tool that works with a template that describes the application's measurement points. The easiest way to understand the required form of the template is to have oml2-scaffold generate a default template for the application with the following few step.
     9
     10{{{
     11$ oml2-scaffold --app generator
     12INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
     13INFO    Created generator.rb
     14INFO    You can now generate a skeleton main file with '/usr/bin/oml2-scaffold --main generator.rb' and its associated Makefile with '/usr/bin/oml2-scaffold --make generator.rb'
     15}}}
     16
     17oml-scaffold2 uses the application template to generate a complete, working, application skeleton with the necessary headers, injection helper functions, the main() function and Makefile.
     18
     19{{{
     20$ oml2-scaffold --main --make generator.rb
     21INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
     22INFO    Created generator.c and config.h
     23INFO    Created Makefile
     24}}}
     25
     26The application can now be built by running a simple make command (it will take care of using scaffold to generate the needed OML and popt headers):
     27
     28{{{
     29$ make
     30/usr/bin/oml2-scaffold --opts generator.rb
     31INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
     32INFO    Created generator_popt.h
     33/usr/bin/oml2-scaffold --oml generator.rb
     34INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
     35INFO    Created generator_oml.h
     36cc -c -Wall -Werror -g -I.  generator.c -o generator.o
     37cc -o generator generator.o  -loml2 -lpopt
     38}}}
     39
     40At this point we have a complete, working, OML application. The skeleton application can be run as follows:
     41
     42{{{
     43$ ./generator --oml-id genid --oml-domain gendomain --oml-collect localhost
     44INFO   OML Client V2.10.0 [Protocol V4] Copyright 2007-2013, NICTA
     45WARN   Attempting to send NULL or empty blob; blob of length 0 will be sent
     46...
     47^CINFO   Waiting for buffered queue reader thread to drain...
     48INFO   Net_stream: attempting to connect to server at tcp://localhost:3003
     49INFO   Buffered queue reader thread finished OK...
     50}}}
     51
     52The application can be terminated by sending it a SIGINT (the easiest way to do so it by hitting Ctrl+C, shown as ^C in the listing above).
     53
     54The WARN line above is expected as the generated code demonstrates the use of every supported OML datatypes, but does not know what to inject. For the blob element, it simply injects a NULL elements, which is probably not what is desired in normal applications, hence the warning. The next few sections now study the generated files more in depth, and shows how to modifie the code to make it do useful injection (and remove this WARN).
     55
     56== Customizing the Application ==
     57The skeleton application generates a fully functional program that you can run and which will record results using OML. The only problem is that it is just an example and will need to be modified to do whatever useful work that you require. In this section we look at the process of customizing the application template so it records what you want and then re-generating the application.
     58
     59== Examining the Application Template ==
     60The application template (named generator.rb in this example) defines the properties and measurement points defined by the application. This file can be used directly by OMF to control the instrumented application and its syntax is documented here. If we examine the file in detail it will be as follows:
     61
     62{{{
     63 1# This file was automatically generated by oml2-scaffold V2.10.0
     64 2# The syntax of this file is documented at [0].
     65 3#
     66 4# [0] http://doc.mytestbed.net/doc/omf/OmfEc/Backward/AppDefinition.html
     67 5
     68 6defApplication('omehani:app:generator', 'generator') do |app|
     69 7
     70 8  app.version(1, 0, 0)
     71 9  app.shortDescription = 'A short description'
     7210  app.description = %{
     7311A longer description describing in more detail what this application
     7412is doing and useful for.
     7513}
     7614  app.path = "/usr/local/bin/generator"
     7715
     7816  # Declare command-line arguments; generate Popt parser with
     7917  #  oml2-scaffold --opts generator.rb
     8018  app.defProperty('loop', 'Create periodic result', '-l',
     8119        :type => 'boolean', :mnemonic => 'l')
     8220  app.defProperty('delay', 'Delay between consecutive measurements', '-d',
     8321        :type => 'integer', :unit => 'seconds', :mnemonic => 'd')
     8422
     8523  # Example of all supported command-line argument types; see popt(3) for more details
     8624  app.defProperty('boolean_var', 'Command line option of type boolean', '--boolean',
     8725        :type => :boolean, :var_name => 'varboolean')
     8826  app.defProperty('string_var', 'Command line option of type string', '--string',
     8927        :type => :string, :var_name => 'varstring')
     9028  app.defProperty('long_var', 'Command line option of type long', '--long',
     9129        :type => :long, :var_name => 'varlong')
     9230  app.defProperty('int32_var', 'Command line option of type int32', '--int32',
     9331        :type => :int32, :var_name => 'varint32')
     9432  app.defProperty('uint32_var', 'Command line option of type uint32', '--uint32',
     9533        :type => :uint32, :var_name => 'varuint32')
     9634  app.defProperty('int64_var', 'Command line option of type int64', '--int64',
     9735        :type => :int64, :var_name => 'varint64')
     9836  app.defProperty('uint64_var', 'Command line option of type uint64', '--uint64',
     9937        :type => :uint64, :var_name => 'varuint64')
     10038  app.defProperty('double_var', 'Command line option of type double', '--double',
     10139        :type => :double, :var_name => 'vardouble')
     10240
     10341  # Declare measurement points; generate OML injection helpers with
     10442  #  oml2-scaffold --oml generator.rb
     10543  app.defMeasurement("sensor") do |mp|
     10644    mp.defMetric('val', :int32)
     10745    mp.defMetric('inverse', :double)
     10846    mp.defMetric('name', :string)
     10947  end
     11048
     11149  # Declare a giant Measurement Point showing all supported types
     11250  app.defMeasurement("example") do |mp|
     11351    mp.defMetric('boolean_field', :boolean)
     11452    mp.defMetric('string_field', :string)
     11553    mp.defMetric('int32_field', :int32)
     11654    mp.defMetric('uint32_field', :uint32)
     11755    mp.defMetric('int64_field', :int64)
     11856    mp.defMetric('uint64_field', :uint64)
     11957    mp.defMetric('double_field', :double)
     12058    mp.defMetric('blob_field', :blob)
     12159    mp.defMetric('guid_field', :guid)
     12260  end
     12361
     12462end
     12563
     12664# Local Variables:
     12765# mode:ruby
     12866# End:
     12967# vim: ft=ruby:sw=2
     130}}}
     131
     132As you can see the application template is defined using Ruby syntax but you shouldn’t need to know Ruby in order to define applications using oml2-scaffold. All of the most useful features of the template language are illustrated in the example above. It should be sufficient to copy and paste the example and then modify it to suit your own needs. The structure of the generated template file is as follows:
     133
     134=== General Properties (lines 6 to 14) ===
     135
     136* line 6: Define a new OML-capable application
     137* line 8: Defines the version of this particular application
     138* line 9: The short description of this application
     139* lines 10-13: A more complete description of the application
     140* line 14: The path where this application is to be installed (for OMF to find it)
     141
     142=== Command line options (lines 16 to 39) ===
     143
     144* lines 16-39: Define the properties that would be tunable from the command line. The syntax to pass this properties via the command line is either "--Property_Name" or "-N". :mnemonic is use to define the one-letter shorthand if needed.
     145
     146=== Measurements (lines 41 to 60) ===
     147
     148* lines 41-60: Defines two OML measurement points, the second shows all supported types
     149
     150== Customizing the Application Template ==
     151
     152The next step is to edit application template so that it describes the desired command line options and measurement points for your application. We do this by starting our favorite text editor and changing the file so that it specifies our desired measurement points.
     153In our example we know what the generator example application should look like and so we simply edit generator.rb until it looks like this:
     154
     155{{{
     156 1# This file was automatically generated by oml2-scaffold V2.10.0
     157 2# The syntax of this file is documented at [0].
     158 3#
     159 4# [0] http://doc.mytestbed.net/doc/omf/OmfEc/Backward/AppDefinition.html
     160 5
     161 6defApplication('omehani:app:generator', 'generator') do |app|
     162 7
     163 8  app.version(1, 0, 0)
     164 9  app.shortDescription = 'Example OML application'
     16510  app.description = %{
     16611This programs implements a simple sine wave generator with
     16712the output measured by OML.
     16813
     16914It is almost only generated by oml2-scaffold.
     17015}
     17116  app.path = "/usr/local/bin/generator"
     17217
     17318  # Declare command-line arguments; generate Popt parser with
     17419  #  oml2-scaffold --opts generator.rb
     17520  app.defProperty('amplitude', 'Amplitude of produced signal', '-a',
     17621        :type => :float, :mnemonic => 'a')
     17722  app.defProperty('frequency', 'Frequency of wave generated', '-f',
     17823        :type => :float, :unit => 'hertz')
     17924  app.defProperty('samples', 'Number of samples to take. -1 = forever', '-n',
     18025        :type => :int32)
     18126  app.defProperty('sample-interval', 'Time between consecutive measurements', '-i',
     18227        :type => :float, :unit => 'seconds',
     18328        :var_name => "sample_interval") # This can be omitted, as scaffold will do the cleanup a needed.
     18429
     18530  # Declare measurement points; generate OML injection helpers with
     18631  #  oml2-scaffold --oml generator.rb
     18732  # Declare measurement points; generate OML injection helpers with
     18833  #  oml2-scaffold --oml generator.rb
     18934  app.defMeasurement("d_lin") do |m|
     19035    m.defMetric('label', :string)
     19136    m.defMetric('seq_no', :uint32)
     19237  end
     19338
     19439  app.defMeasurement("d_sin") do |m|
     19540    m.defMetric('label', :string)
     19641    m.defMetric('phase', :double)
     19742    m.defMetric('value', :double)
     19843  end
     19944
     20045end
     20146
     20247# Local Variables:
     20348# mode:ruby
     20449# End:
     20550# vim: ft=ruby:sw=2
     206}}}
     207
     208In this example we have changed the following lines (and removed some comments for legibility).
     209
     210* line 9: We give the short description of this application
     211* lines 10-15: We provide a more complete description of the application
     212* lines 18-28: We give the properties that are tunable for the execution of the application. In this example application we want to be able to control the characteristics of the generated wave from the command line.
     213* lines 30-43: We define the OML measurement points for this application.
     214
     215=== Re-Generating the Skeleton Application ===
     216Once the application template is complete we are now ready to re-generate the code. We do so by executing oml2-scaffold as follows:
     217
     218{{{
     219$ oml2-scaffold --make --main --force generator.rb
     220INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
     221INFO    Created Makefile
     222INFO    Created generator.c and config.h
     223}}}
     224
     225This oml2-scaffold command creates a number of files: the main source code, generator.c, a skeleton config.h containing version information (it can be replaced with an autoconf-generated config.h), and a Makefile taking care of:
     226
     2271. generating support headers files generator_oml.h and generator_popt.h defining OML measurement points and helper functions, and command-line options-parsing code, respectively; both these files are also generated using oml2-scaffold, but it is not recommended to keep static versions, rather, they should be regenerated on every build, which is what the Makefile does;
     2281. compiling the generator into an executable binary (the standard CFLAGS, LDFLAGS and LIBS variables, as well as DESTDIR are recognised).
     229
     230=== Main file (<appname.c>) ===
     231
     232The most important of these generated files is named <appname>.c (generator.c in our example). It contains the top-level main() declaration for the application, as well as a token run() function which needs be adapted to implement the desired behaviour and measurements of the application. For our example this generates a file named generator.c containing the following code.
     233
     234{{{
     235 1/*
     236 2 * This file was automatically generated by oml2-scaffold V2.10.0
     237 3 * for generator version 1.0.0.
     238 4 * Please edit to suit your needs; the run() function should contain application code.
     239 5 */
     240 6#include <stdio.h>
     241 7#include <math.h>
     242 8#include <unistd.h>
     243 9
     24410#include <popt.h>
     24511#include <oml2/omlc.h>
     24612
     24713#define USE_OPTS /* Include command line parsing code*/
     24814#include "generator_popt.h"
     24915
     25016#define OML_FROM_MAIN /* Define storage for some global variables; #define this in only one file */
     25117#include "generator_oml.h"
     25218
     25319#include "config.h"
     25420
     25521/* Do application-specific work here */
     25622void
     25723run(opts_t* opts, oml_mps_t* oml_mps)
     25824{
     25925  double angle = 0;
     26026  double delta = opts->frequency * opts->sample_interval * 2 * M_PI;
     26127  unsigned long sleep = (unsigned long)(opts->sample_interval * 1E6);
     26228
     26329  printf("%f, %f, %ld\n", M_PI, delta, sleep);
     26430
     26531  int i = opts->samples;
     26632  unsigned int count = 1;
     26733  for (; i != 0; i--, count++) {
     26834    char label[64];
     26935    sprintf(label, "sample-%d", count);
     27036
     27137    oml_inject_d_lin(oml_mps->d_lin, label, count);
     27238
     27339    double value = opts->amplitude * sin(angle);
     27440    oml_inject_d_sin(oml_mps->d_sin, label, angle, value);
     27541
     27642    printf("%s %d | %f %f\n", label, count, angle, value);
     27743
     27844    angle = fmodf(angle + delta, 2 * M_PI);
     27945    usleep(sleep);
     28046  }
     28147}
     28248
     28349int
     28450main(int argc, const char *argv[])
     28551{
     28652  int c, ret;
     28753
     28854  if((ret = omlc_init("generator", &argc, argv, NULL)) < 0) {
     28955    logerror("Could not initialise OML\n");
     29056    return -1;
     29157  }
     29258
     29359  /* Parse command line arguments */
     29460  poptContext optCon = poptGetContext(NULL, argc, argv, options, 0); /* options is defined in generator_popt.h */
     29561  while ((c = poptGetNextOpt(optCon)) > 0) {}
     29662
     29763  /* Initialise measurement points, only if OML is not disabled */
     29864  oml_register_mps(); /* Defined in generator_oml.h */
     29965  if(ret == 0 && omlc_start()) {
     30066    logerror("Could not start OML\n");
     30167    return -1;
     30268  }
     30369
     30470  run(g_opts, g_oml_mps_generator); /* Do some work and injections, see above */
     30571
     30672  omlc_close();
     30773
     30874  return 0;
     30975}
     31076
     31177/*
     31278 Local Variables:
     31379 mode: C
     31480 tab-width: 2
     31581 indent-tabs-mode: nil
     31682 End:
     31783 vim: sw=2:sts=2:expandtab
     31884*/
     319}}}
     320
     321In this program, OML is initialised at the line 54. The basic option-parsing logic for the declared properties (but not the --oml-* options, which are directly handled by the OML client library) is handled through popt(3) on lines 59--61. Line 64 use the Measurement Point registration helper (which is defined in the generator_oml.h@ file when OML_FROM_MAIN is #define@d). The OML infrastructure is started on line 65. Finally the measurement process is started at line 70 by calling the @run() function. The default version of this function loops over the process of generating a measurement and then sleeping. It is this function which you will need to edit to perform the actual work of the application.
     322
     323This main file includes three local header files
     324
     325* generator_oml.h, containing declaration of OML measurement points and helper functions (such as oml_register_mps and oml_inject_d_lin(), above);
     326* generator_popt.h, containing code to configure popt to parse the command line options (see the call to poptGetNextOpt() in the main() function;
     327* config.h, containing autoconf-like version information.
     328
     329While it is safe to keep a hardcoded version of config.h, the <appname>_oml.h and <appname>_popt.h should not be modified nor checked in into any source control system. Rather, they should always be regenerated at build time. This make application code more portable acMaross versions.
     330
     331Next, we review the contents of the <appname>_oml.h file. While it is not to be modified in most cases, some application writers might need to use low-level OML API calls which are usually hidden in this header. This might particularly be the case for applications for which measurement points are not known in advance (e.g., plugin-based systems).
     332
     333=== OML Declaractions (<appname>_oml.h) ===
     334The oml2-scaffold command also generates the code for registering and injecting values at the Measurement Points based on the defMeasurement description in the application template (using the --oml command line option). These are defined in the file <appname>_oml.h. For our example, generator_oml.h ontains the following code.
     335
     336{{{
     337  1/*
     338  2 * This file was automatically generated by oml2-scaffold V2.10.0
     339  3 * for generator version 1.0.0.
     340  4 * Please do not edit.
     341  5 */
     342  6
     343  7#ifndef GENERATOR_OML_H
     344  8#define GENERATOR_OML_H
     345  9
     346 10#ifdef __cplusplus
     347 11extern "C" {
     348 12#endif
     349 13
     350 14/* Get size_t and NULL from <stddef.h>.  */
     351 15#include <stddef.h>
     352 16
     353 17#include <oml2/omlc.h>
     354 18
     355 19typedef struct {
     356 20  OmlMP* d_lin;
     357 21  OmlMP* d_sin;
     358 22
     359 23} oml_mps_t;
     360 24
     361 25#ifdef OML_FROM_MAIN
     362 26/*
     363 27 * Only declare storage once, usually from the main
     364 28 * source, where OML_FROM_MAIN is defined
     365 29 */
     366 30
     367 31static OmlMPDef oml_d_lin_def[] = {
     368 32  {"label", OML_STRING_VALUE},
     369 33  {"seq_no", OML_UINT32_VALUE},
     370 34  {NULL, (OmlValueT)0}
     371 35};
     372 36
     373 37static OmlMPDef oml_d_sin_def[] = {
     374 38  {"label", OML_STRING_VALUE},
     375 39  {"phase", OML_DOUBLE_VALUE},
     376 40  {"value", OML_DOUBLE_VALUE},
     377 41  {NULL, (OmlValueT)0}
     378 42};
     379 43
     380 44static oml_mps_t g_oml_mps_storage;
     381 45oml_mps_t* g_oml_mps_generator = &g_oml_mps_storage;
     382 46
     383 47static inline void
     384 48oml_register_mps()
     385 49{
     386 50  g_oml_mps_generator->d_lin = omlc_add_mp("d_lin", oml_d_lin_def);
     387 51  g_oml_mps_generator->d_sin = omlc_add_mp("d_sin", oml_d_sin_def);
     388 52
     389 53}
     390 54
     391 55#else
     392 56/*
     393 57 * Not included from the main source, only declare the global pointer
     394 58 * to the MPs and injection helpers.
     395 59 */
     396 60
     397 61extern oml_mps_t* g_oml_mps_generator;
     398 62
     399 63#endif /* OML_FROM_MAIN */
     400 64
     401 65static inline void
     402 66oml_inject_d_lin(OmlMP* mp, const char* label, uint32_t seq_no)
     403 67{
     404 68  OmlValueU v[2];
     405 69
     406 70  omlc_zero_array(v, 2);
     407 71
     408 72  omlc_set_string(v[0], label);
     409 73  omlc_set_uint32(v[1], seq_no);
     410 74
     411 75  omlc_inject(mp, v);
     412 76
     413 77  omlc_reset_string(v[0]);
     414 78}
     415 79
     416 80static inline void
     417 81oml_inject_d_sin(OmlMP* mp, const char* label, double phase, double value)
     418 82{
     419 83  OmlValueU v[3];
     420 84
     421 85  omlc_zero_array(v, 3);
     422 86
     423 87  omlc_set_string(v[0], label);
     424 88  omlc_set_double(v[1], phase);
     425 89  omlc_set_double(v[2], value);
     426 90
     427 91  omlc_inject(mp, v);
     428 92
     429 93  omlc_reset_string(v[0]);
     430 94}
     431 95
     432 96/* Compatibility with old applications */
     433 97#ifndef g_oml_mps
     434 98# define g_oml_mps    g_oml_mps_generator
     435 99#endif
     436100
     437101#ifdef __cplusplus
     438102}
     439103#endif
     440104
     441105#endif /* GENERATOR_OML_H */
     442}}}
     443
     444There a three important sections in this file, related to
     445
     4461. declaration,
     4471. registration, and
     4481. injection in Measurement Points.
     449
     450* lines 31--42 define two OmlMPDef describing the MPs;
     451* lines 44--45 prepares an area where to store the MP handler returned by omlc_add_mp ;
     452* lines 47--53 contains the oml_register_mps helper, using omlc_add_mp(3) to register both MPs with the OML library;
     453* lines 66--95 containc the oml_inject_MPNAME injection helpers for both MP.
     454
     455The injection helpers deserver a closer look. They manipulate OmlValueU arrays, in which the data to inject is packed. These arrays need to be initialised before any use, with the omlc_zero_array function. Variables are then packed into the array with the omlc_set_TYPE family of functions. Once done, the array can then be injected into the desired MP handler with omlc_inject, and the OmlValueUs cleaned up with omlc_reset_TYPE if they used some dynamic storage (as of 2.10.0, this is the case for strings and blobs).
     456
     457It is worthy of notice that, as OmlValueU(3) are untyped, omlc_set_TYPE(3) cannot do any type checking. Moreover, dynamic memory allocation, and the necessary dealloctaions depend on the data type of a specific OmlValueU(3) in the array. Therefore, unless there is a good reason to manually write this injection code, it is highly recommended to let oml2-scaffold do it, and rely on its typed oml_inject_MPNAME helpers.
     458
     459=== Command-line Parsing Logic (<appname>_popt.h) ===
     460
     461The oml2-scaffold command can also generate popt code to parse the command line (using the --opt command line option). These are defined in the file <appname>_popt.h. There is no reason to modify this file.
     462
     463== Building and Running the Modified Application ==
     464
     465The application can now be rebuilt with a simple make and executed with a simple
     466
     467{{{
     468$ ./generator --amplitude 42 --frequency 1337 --samples 1024 --sample-interval .25 \
     469      --oml-id genid --oml-domain gendomain --oml-collect localhost
     470}}}
     471
     472This time we should see the measurement points stored in the database that are specific to our generator application.
     473
     474== Instrumenting an Existing Application ==
     475
     476Once again, oml2-scaffold can be of assistance. Although an existing application will already have its own Makefile and main() function the initialisation code generated by oml2-scaffold (lines 54--57 and 63--67 in <appname>.c) can be cut and pasted into the existing application at the appropriate points, and the <appname>_oml.h #include'd where needed and generated at build time, so the helpers are available.