Changes between Initial Version and Version 1 of General/2aTutorials

Timestamp:

Feb 18, 2019, 10:33:12 PM (6 years ago)

Author:

seskar

Comment:

--

General/2aTutorials

@@ +1 @@
+= Quick Start Tutorial =
+
+[[TOC(heading=General Documentation, General/*, depth=1)]]
+
+The easiest way to instrument or create an OML application is by using oml2-scaffold. Starting from a description of the application's measurement points it can rapidly generate the skeleton of the OML application. The aim of this tool is to facilitate the generation of type-safe C code. Previous versions of this tutorial make use of the low-level C API functions and data structures. Although this approach is still valid, its use is discouraged because the low-level API lacks type-safety. For new applications the use of oml2-scaffold is preferred. oml2-scaffold is a standard part of the OML release. Under Debian, for example, this tool is available in the liboml2-dev package.
+
+== Using OML scaffold to Create the Skeleton of the Application ==
+In this guide we will use the OML scaffold to re-create the "sine generator" example application which is shipped with the OML source (source:example/liboml2). This simple example demonstrates both an OML-instrumented application as well as the code-generating capabilities of oml2-scaffold. oml2-scaffold itself is a Ruby tool that works with a template that describes the application's measurement points. The easiest way to understand the required form of the template is to have oml2-scaffold generate a default template for the application with the following few step.
+
+{{{
+$ oml2-scaffold --app generator
+INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
+INFO    Created generator.rb
+INFO    You can now generate a skeleton main file with '/usr/bin/oml2-scaffold --main generator.rb' and its associated Makefile with '/usr/bin/oml2-scaffold --make generator.rb'
+}}}
+
+oml-scaffold2 uses the application template to generate a complete, working, application skeleton with the necessary headers, injection helper functions, the main() function and Makefile.
+
+{{{
+$ oml2-scaffold --main --make generator.rb
+INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
+INFO    Created generator.c and config.h
+INFO    Created Makefile
+}}}
+
+The application can now be built by running a simple make command (it will take care of using scaffold to generate the needed OML and popt headers):
+
+{{{
+$ make
+/usr/bin/oml2-scaffold --opts generator.rb
+INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
+INFO    Created generator_popt.h
+/usr/bin/oml2-scaffold --oml generator.rb
+INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
+INFO    Created generator_oml.h
+cc -c -Wall -Werror -g -I.  generator.c -o generator.o
+cc -o generator generator.o  -loml2 -lpopt
+}}}
+
+At this point we have a complete, working, OML application. The skeleton application can be run as follows:
+
+{{{
+$ ./generator --oml-id genid --oml-domain gendomain --oml-collect localhost
+INFO   OML Client V2.10.0 [Protocol V4] Copyright 2007-2013, NICTA
+WARN   Attempting to send NULL or empty blob; blob of length 0 will be sent
+...
+^CINFO   Waiting for buffered queue reader thread to drain...
+INFO   Net_stream: attempting to connect to server at tcp://localhost:3003
+INFO   Buffered queue reader thread finished OK...
+}}}
+
+The application can be terminated by sending it a SIGINT (the easiest way to do so it by hitting Ctrl+C, shown as ^C in the listing above).
+
+The WARN line above is expected as the generated code demonstrates the use of every supported OML datatypes, but does not know what to inject. For the blob element, it simply injects a NULL elements, which is probably not what is desired in normal applications, hence the warning. The next few sections now study the generated files more in depth, and shows how to modifie the code to make it do useful injection (and remove this WARN).
+
+== Customizing the Application ==
+The skeleton application generates a fully functional program that you can run and which will record results using OML. The only problem is that it is just an example and will need to be modified to do whatever useful work that you require. In this section we look at the process of customizing the application template so it records what you want and then re-generating the application.
+
+== Examining the Application Template ==
+The application template (named generator.rb in this example) defines the properties and measurement points defined by the application. This file can be used directly by OMF to control the instrumented application and its syntax is documented here. If we examine the file in detail it will be as follows:
+
+{{{
+ 1# This file was automatically generated by oml2-scaffold V2.10.0
+ 2# The syntax of this file is documented at [0].
+ 3#
+ 4# [0] http://doc.mytestbed.net/doc/omf/OmfEc/Backward/AppDefinition.html
+ 5
+ 6defApplication('omehani:app:generator', 'generator') do |app|
+ 7
+ 8  app.version(1, 0, 0)
+ 9  app.shortDescription = 'A short description'
+10  app.description = %{
+11A longer description describing in more detail what this application
+12is doing and useful for.
+13}
+14  app.path = "/usr/local/bin/generator" 
+15
+16  # Declare command-line arguments; generate Popt parser with
+17  #  oml2-scaffold --opts generator.rb
+18  app.defProperty('loop', 'Create periodic result', '-l',
+19        :type => 'boolean', :mnemonic => 'l')
+20  app.defProperty('delay', 'Delay between consecutive measurements', '-d',
+21        :type => 'integer', :unit => 'seconds', :mnemonic => 'd')
+22
+23  # Example of all supported command-line argument types; see popt(3) for more details
+24  app.defProperty('boolean_var', 'Command line option of type boolean', '--boolean',
+25        :type => :boolean, :var_name => 'varboolean')
+26  app.defProperty('string_var', 'Command line option of type string', '--string',
+27        :type => :string, :var_name => 'varstring')
+28  app.defProperty('long_var', 'Command line option of type long', '--long',
+29        :type => :long, :var_name => 'varlong')
+30  app.defProperty('int32_var', 'Command line option of type int32', '--int32',
+31        :type => :int32, :var_name => 'varint32')
+32  app.defProperty('uint32_var', 'Command line option of type uint32', '--uint32',
+33        :type => :uint32, :var_name => 'varuint32')
+34  app.defProperty('int64_var', 'Command line option of type int64', '--int64',
+35        :type => :int64, :var_name => 'varint64')
+36  app.defProperty('uint64_var', 'Command line option of type uint64', '--uint64',
+37        :type => :uint64, :var_name => 'varuint64')
+38  app.defProperty('double_var', 'Command line option of type double', '--double',
+39        :type => :double, :var_name => 'vardouble')
+40
+41  # Declare measurement points; generate OML injection helpers with
+42  #  oml2-scaffold --oml generator.rb
+43  app.defMeasurement("sensor") do |mp|
+44    mp.defMetric('val', :int32)
+45    mp.defMetric('inverse', :double)
+46    mp.defMetric('name', :string)
+47  end
+48
+49  # Declare a giant Measurement Point showing all supported types
+50  app.defMeasurement("example") do |mp|
+51    mp.defMetric('boolean_field', :boolean)
+52    mp.defMetric('string_field', :string)
+53    mp.defMetric('int32_field', :int32)
+54    mp.defMetric('uint32_field', :uint32)
+55    mp.defMetric('int64_field', :int64)
+56    mp.defMetric('uint64_field', :uint64)
+57    mp.defMetric('double_field', :double)
+58    mp.defMetric('blob_field', :blob)
+59    mp.defMetric('guid_field', :guid)
+60  end
+61
+62end
+63
+64# Local Variables:
+65# mode:ruby
+66# End:
+67# vim: ft=ruby:sw=2
+}}}
+
+As you can see the application template is defined using Ruby syntax but you shouldn’t need to know Ruby in order to define applications using oml2-scaffold. All of the most useful features of the template language are illustrated in the example above. It should be sufficient to copy and paste the example and then modify it to suit your own needs. The structure of the generated template file is as follows:
+
+=== General Properties (lines 6 to 14) ===
+
+* line 6: Define a new OML-capable application
+* line 8: Defines the version of this particular application
+* line 9: The short description of this application
+* lines 10-13: A more complete description of the application
+* line 14: The path where this application is to be installed (for OMF to find it)
+
+=== Command line options (lines 16 to 39) ===
+
+* lines 16-39: Define the properties that would be tunable from the command line. The syntax to pass this properties via the command line is either "--Property_Name" or "-N". :mnemonic is use to define the one-letter shorthand if needed.
+
+=== Measurements (lines 41 to 60) ===
+
+* lines 41-60: Defines two OML measurement points, the second shows all supported types
+
+== Customizing the Application Template ==
+
+The next step is to edit application template so that it describes the desired command line options and measurement points for your application. We do this by starting our favorite text editor and changing the file so that it specifies our desired measurement points.
+In our example we know what the generator example application should look like and so we simply edit generator.rb until it looks like this:
+
+{{{
+ 1# This file was automatically generated by oml2-scaffold V2.10.0
+ 2# The syntax of this file is documented at [0].
+ 3#
+ 4# [0] http://doc.mytestbed.net/doc/omf/OmfEc/Backward/AppDefinition.html
+ 5
+ 6defApplication('omehani:app:generator', 'generator') do |app|
+ 7
+ 8  app.version(1, 0, 0)
+ 9  app.shortDescription = 'Example OML application'
+10  app.description = %{
+11This programs implements a simple sine wave generator with
+12the output measured by OML.
+13
+14It is almost only generated by oml2-scaffold.
+15}
+16  app.path = "/usr/local/bin/generator" 
+17
+18  # Declare command-line arguments; generate Popt parser with
+19  #  oml2-scaffold --opts generator.rb
+20  app.defProperty('amplitude', 'Amplitude of produced signal', '-a',
+21        :type => :float, :mnemonic => 'a')
+22  app.defProperty('frequency', 'Frequency of wave generated', '-f',
+23        :type => :float, :unit => 'hertz')
+24  app.defProperty('samples', 'Number of samples to take. -1 = forever', '-n',
+25        :type => :int32)
+26  app.defProperty('sample-interval', 'Time between consecutive measurements', '-i',
+27        :type => :float, :unit => 'seconds',
+28        :var_name => "sample_interval") # This can be omitted, as scaffold will do the cleanup a needed.
+29
+30  # Declare measurement points; generate OML injection helpers with
+31  #  oml2-scaffold --oml generator.rb
+32  # Declare measurement points; generate OML injection helpers with
+33  #  oml2-scaffold --oml generator.rb
+34  app.defMeasurement("d_lin") do |m|
+35    m.defMetric('label', :string)
+36    m.defMetric('seq_no', :uint32)
+37  end
+38
+39  app.defMeasurement("d_sin") do |m|
+40    m.defMetric('label', :string)
+41    m.defMetric('phase', :double)
+42    m.defMetric('value', :double)
+43  end
+44
+45end
+46
+47# Local Variables:
+48# mode:ruby
+49# End:
+50# vim: ft=ruby:sw=2
+}}}
+
+In this example we have changed the following lines (and removed some comments for legibility).
+
+* line 9: We give the short description of this application
+* lines 10-15: We provide a more complete description of the application
+* lines 18-28: We give the properties that are tunable for the execution of the application. In this example application we want to be able to control the characteristics of the generated wave from the command line.
+* lines 30-43: We define the OML measurement points for this application.
+
+=== Re-Generating the Skeleton Application ===
+Once the application template is complete we are now ready to re-generate the code. We do so by executing oml2-scaffold as follows:
+
+{{{
+$ oml2-scaffold --make --main --force generator.rb
+INFO    oml2-scaffold V2.10.0 Copyright 2009-2013, NICTA
+INFO    Created Makefile
+INFO    Created generator.c and config.h
+}}}
+
+This oml2-scaffold command creates a number of files: the main source code, generator.c, a skeleton config.h containing version information (it can be replaced with an autoconf-generated config.h), and a Makefile taking care of:
+
+1. generating support headers files generator_oml.h and generator_popt.h defining OML measurement points and helper functions, and command-line options-parsing code, respectively; both these files are also generated using oml2-scaffold, but it is not recommended to keep static versions, rather, they should be regenerated on every build, which is what the Makefile does;
+1. compiling the generator into an executable binary (the standard CFLAGS, LDFLAGS and LIBS variables, as well as DESTDIR are recognised).
+
+=== Main file (<appname.c>) ===
+
+The most important of these generated files is named <appname>.c (generator.c in our example). It contains the top-level main() declaration for the application, as well as a token run() function which needs be adapted to implement the desired behaviour and measurements of the application. For our example this generates a file named generator.c containing the following code.
+
+{{{
+ 1/*
+ 2 * This file was automatically generated by oml2-scaffold V2.10.0
+ 3 * for generator version 1.0.0.
+ 4 * Please edit to suit your needs; the run() function should contain application code.
+ 5 */
+ 6#include <stdio.h>
+ 7#include <math.h>
+ 8#include <unistd.h>
+ 9
+10#include <popt.h>
+11#include <oml2/omlc.h>
+12
+13#define USE_OPTS /* Include command line parsing code*/
+14#include "generator_popt.h" 
+15
+16#define OML_FROM_MAIN /* Define storage for some global variables; #define this in only one file */
+17#include "generator_oml.h" 
+18
+19#include "config.h" 
+20
+21/* Do application-specific work here */
+22void
+23run(opts_t* opts, oml_mps_t* oml_mps)
+24{
+25  double angle = 0;
+26  double delta = opts->frequency * opts->sample_interval * 2 * M_PI;
+27  unsigned long sleep = (unsigned long)(opts->sample_interval * 1E6);
+28
+29  printf("%f, %f, %ld\n", M_PI, delta, sleep);
+30
+31  int i = opts->samples;
+32  unsigned int count = 1;
+33  for (; i != 0; i--, count++) {
+34    char label[64];
+35    sprintf(label, "sample-%d", count);
+36
+37    oml_inject_d_lin(oml_mps->d_lin, label, count);
+38
+39    double value = opts->amplitude * sin(angle);
+40    oml_inject_d_sin(oml_mps->d_sin, label, angle, value);
+41
+42    printf("%s %d | %f %f\n", label, count, angle, value);
+43
+44    angle = fmodf(angle + delta, 2 * M_PI);
+45    usleep(sleep);
+46  }
+47}
+48
+49int
+50main(int argc, const char *argv[])
+51{
+52  int c, ret;
+53
+54  if((ret = omlc_init("generator", &argc, argv, NULL)) < 0) {
+55    logerror("Could not initialise OML\n");
+56    return -1;
+57  }
+58
+59  /* Parse command line arguments */
+60  poptContext optCon = poptGetContext(NULL, argc, argv, options, 0); /* options is defined in generator_popt.h */
+61  while ((c = poptGetNextOpt(optCon)) > 0) {}
+62
+63  /* Initialise measurement points, only if OML is not disabled */
+64  oml_register_mps(); /* Defined in generator_oml.h */
+65  if(ret == 0 && omlc_start()) {
+66    logerror("Could not start OML\n");
+67    return -1;
+68  }
+69
+70  run(g_opts, g_oml_mps_generator); /* Do some work and injections, see above */
+71
+72  omlc_close();
+73
+74  return 0;
+75}
+76
+77/*
+78 Local Variables:
+79 mode: C
+80 tab-width: 2
+81 indent-tabs-mode: nil
+82 End:
+83 vim: sw=2:sts=2:expandtab
+84*/
+}}}
+
+In this program, OML is initialised at the line 54. The basic option-parsing logic for the declared properties (but not the --oml-* options, which are directly handled by the OML client library) is handled through popt(3) on lines 59--61. Line 64 use the Measurement Point registration helper (which is defined in the generator_oml.h@ file when OML_FROM_MAIN is #define@d). The OML infrastructure is started on line 65. Finally the measurement process is started at line 70 by calling the @run() function. The default version of this function loops over the process of generating a measurement and then sleeping. It is this function which you will need to edit to perform the actual work of the application.
+
+This main file includes three local header files
+
+* generator_oml.h, containing declaration of OML measurement points and helper functions (such as oml_register_mps and oml_inject_d_lin(), above);
+* generator_popt.h, containing code to configure popt to parse the command line options (see the call to poptGetNextOpt() in the main() function;
+* config.h, containing autoconf-like version information.
+
+While it is safe to keep a hardcoded version of config.h, the <appname>_oml.h and <appname>_popt.h should not be modified nor checked in into any source control system. Rather, they should always be regenerated at build time. This make application code more portable acMaross versions.
+
+Next, we review the contents of the <appname>_oml.h file. While it is not to be modified in most cases, some application writers might need to use low-level OML API calls which are usually hidden in this header. This might particularly be the case for applications for which measurement points are not known in advance (e.g., plugin-based systems).
+
+=== OML Declaractions (<appname>_oml.h) ===
+The oml2-scaffold command also generates the code for registering and injecting values at the Measurement Points based on the defMeasurement description in the application template (using the --oml command line option). These are defined in the file <appname>_oml.h. For our example, generator_oml.h ontains the following code.
+
+{{{
+  1/*
+  2 * This file was automatically generated by oml2-scaffold V2.10.0
+  3 * for generator version 1.0.0.
+  4 * Please do not edit.
+  5 */
+  6
+  7#ifndef GENERATOR_OML_H
+  8#define GENERATOR_OML_H
+  9
+ 10#ifdef __cplusplus
+ 11extern "C" {
+ 12#endif
+ 13
+ 14/* Get size_t and NULL from <stddef.h>.  */
+ 15#include <stddef.h>
+ 16
+ 17#include <oml2/omlc.h>
+ 18
+ 19typedef struct {
+ 20  OmlMP* d_lin;
+ 21  OmlMP* d_sin;
+ 22
+ 23} oml_mps_t;
+ 24
+ 25#ifdef OML_FROM_MAIN
+ 26/*
+ 27 * Only declare storage once, usually from the main
+ 28 * source, where OML_FROM_MAIN is defined
+ 29 */
+ 30
+ 31static OmlMPDef oml_d_lin_def[] = {
+ 32  {"label", OML_STRING_VALUE},
+ 33  {"seq_no", OML_UINT32_VALUE},
+ 34  {NULL, (OmlValueT)0}
+ 35};
+ 36
+ 37static OmlMPDef oml_d_sin_def[] = {
+ 38  {"label", OML_STRING_VALUE},
+ 39  {"phase", OML_DOUBLE_VALUE},
+ 40  {"value", OML_DOUBLE_VALUE},
+ 41  {NULL, (OmlValueT)0}
+ 42};
+ 43
+ 44static oml_mps_t g_oml_mps_storage;
+ 45oml_mps_t* g_oml_mps_generator = &g_oml_mps_storage;
+ 46
+ 47static inline void
+ 48oml_register_mps()
+ 49{
+ 50  g_oml_mps_generator->d_lin = omlc_add_mp("d_lin", oml_d_lin_def);
+ 51  g_oml_mps_generator->d_sin = omlc_add_mp("d_sin", oml_d_sin_def);
+ 52
+ 53}
+ 54
+ 55#else
+ 56/*
+ 57 * Not included from the main source, only declare the global pointer
+ 58 * to the MPs and injection helpers.
+ 59 */
+ 60
+ 61extern oml_mps_t* g_oml_mps_generator;
+ 62
+ 63#endif /* OML_FROM_MAIN */
+ 64
+ 65static inline void
+ 66oml_inject_d_lin(OmlMP* mp, const char* label, uint32_t seq_no)
+ 67{
+ 68  OmlValueU v[2];
+ 69
+ 70  omlc_zero_array(v, 2);
+ 71
+ 72  omlc_set_string(v[0], label);
+ 73  omlc_set_uint32(v[1], seq_no);
+ 74
+ 75  omlc_inject(mp, v);
+ 76
+ 77  omlc_reset_string(v[0]);
+ 78}
+ 79
+ 80static inline void
+ 81oml_inject_d_sin(OmlMP* mp, const char* label, double phase, double value)
+ 82{
+ 83  OmlValueU v[3];
+ 84
+ 85  omlc_zero_array(v, 3);
+ 86
+ 87  omlc_set_string(v[0], label);
+ 88  omlc_set_double(v[1], phase);
+ 89  omlc_set_double(v[2], value);
+ 90
+ 91  omlc_inject(mp, v);
+ 92
+ 93  omlc_reset_string(v[0]);
+ 94}
+ 95
+ 96/* Compatibility with old applications */
+ 97#ifndef g_oml_mps
+ 98# define g_oml_mps    g_oml_mps_generator
+ 99#endif
+100
+101#ifdef __cplusplus
+102}
+103#endif
+104
+105#endif /* GENERATOR_OML_H */
+}}}
+
+There a three important sections in this file, related to
+
+1. declaration,
+1. registration, and
+1. injection in Measurement Points.
+
+* lines 31--42 define two OmlMPDef describing the MPs;
+* lines 44--45 prepares an area where to store the MP handler returned by omlc_add_mp ;
+* lines 47--53 contains the oml_register_mps helper, using omlc_add_mp(3) to register both MPs with the OML library;
+* lines 66--95 containc the oml_inject_MPNAME injection helpers for both MP.
+
+The injection helpers deserver a closer look. They manipulate OmlValueU arrays, in which the data to inject is packed. These arrays need to be initialised before any use, with the omlc_zero_array function. Variables are then packed into the array with the omlc_set_TYPE family of functions. Once done, the array can then be injected into the desired MP handler with omlc_inject, and the OmlValueUs cleaned up with omlc_reset_TYPE if they used some dynamic storage (as of 2.10.0, this is the case for strings and blobs).
+
+It is worthy of notice that, as OmlValueU(3) are untyped, omlc_set_TYPE(3) cannot do any type checking. Moreover, dynamic memory allocation, and the necessary dealloctaions depend on the data type of a specific OmlValueU(3) in the array. Therefore, unless there is a good reason to manually write this injection code, it is highly recommended to let oml2-scaffold do it, and rely on its typed oml_inject_MPNAME helpers.
+
+=== Command-line Parsing Logic (<appname>_popt.h) ===
+
+The oml2-scaffold command can also generate popt code to parse the command line (using the --opt command line option). These are defined in the file <appname>_popt.h. There is no reason to modify this file.
+
+== Building and Running the Modified Application ==
+
+The application can now be rebuilt with a simple make and executed with a simple
+
+{{{
+$ ./generator --amplitude 42 --frequency 1337 --samples 1024 --sample-interval .25 \
+      --oml-id genid --oml-domain gendomain --oml-collect localhost
+}}}
+
+This time we should see the measurement points stored in the database that are specific to our generator application.
+
+== Instrumenting an Existing Application ==
+
+Once again, oml2-scaffold can be of assistance. Although an existing application will already have its own Makefile and main() function the initialisation code generated by oml2-scaffold (lines 54--57 and 63--67 in <appname>.c) can be cut and pasted into the existing application at the appropriate points, and the <appname>_oml.h #include'd where needed and generated at build time, so the helpers are available.