Changes between Initial Version and Version 1 of Developer/4aFilters

Timestamp:

Jul 15, 2020, 3:50:43 PM (4 years ago)

Author:

seskar

Comment:

--

Developer/4aFilters

@@ +1 @@
+= Developing Filters =
+
+''This document is a work in progress''
+
+An OML filter is a component providing some filtering functions complying with an OML API. It accepts one or more samples of one or more values (i.e. fields from MPs) via an (*oml_filter_input) function (usually named sample, and called once per sample), and reports one or more filtered values when queried via an (*oml_filter_output) function (usually named process). It also possesses an internal state (*InstanceData for clarity, which is typedef'd to the actual structure defined in the relevant header). A couple of other helper functions are also defined to allow registering the filter to OML, determining its input and output schemas, and creating instances of the filter.
+
+The API is defined in source:lib/client/oml2/oml_filter.h.
+
+== Structure of a Filter ==
+
+In this section, examples are directly based on source:lib/client/filter/first_filter.c and source:lib/client/filter/first_filter.h. The first filter reports the first data sample from a period.
+
+=== Header ===
+The header file for a filter is where the structure defining its internal state is defined.
+
+The first filter only needs to store the value of the first sample it has seen. However, it also needs to keep track of whether it has seen a sample yet. Therefore, its internal state consists of two elements:
+ * a OmlValue (or more precisely a pointer where) to store the element;
+ * an int which is set to 1 if the first element hasn't been seen during the sampling period yet.
+
+Refer to source:lib/client/filter/first_filter.h for a full header.
+
+=== Implementation ===
+
+The life of a filter is as follows tells OML about its methods and its output schema.
+
+Filter instances can then be created.
+ 1. Creation of a new instance and its related internal data ((*oml_filter_create)),
+ 1. Parameters setting ((*oml_filter_set)),
+ 1. Injection of samples ((*oml_filter_input)),
+ 1. Query for the filtered output ((*oml_filter_output)),
+ 1. Function to start a new sampling period ((*oml_filter_output)).
+
+The registration function is a simple code which calls omlf_register_filter() to tell OML about the methods it provide implementing the OML filter API. It also specifies the name by which the filter will be known from OML clients (e.g. the configuration file). The omlf_register_filter() takes its parameters in the following order:
+
+ 1. const char* filter_name: name of the filter,
+ 1. void* (*oml_filter_create) (OmlValueT type, OmlValue* result): pointer to the instance creation function,
+ 1. int (*oml_filter_set) (struct _omlFilter* filter, const char* name, OmlValue* value): pointer to the filter parameter-setting function,
+ 1. int (*oml_filter_input) (struct _omlFilter* filter, OmlValue* value): pointer to the sample submission function,
+ 1. int (*oml_filter_output) (struct _omlFilter* filter, struct _omlWriter* writer): pointer to the function writing the filtered data (using the given writer),
+ 1. int (*oml_filter_newwindow) (struct _omlFilter* filter): pointer to the function clearing the filter's state to start a new sampling window,
+ 1. int (*oml_filter_meta)(struct _omlFilter* filter, int index_offset, char** namePtr, OmlValueT* type): see below,
+ 1. OmlFilterDef* filter_def: see below.
+
+The two last parameters are used specification of the filter's output schema via either an OmlFilterDef array (a null-terminated list of named fields of a given type), or by using
+
+{{{
+int (*oml_filter_meta)(struct _omlFilter filter, int index_offset, char** namePtr,  OmlValueT* type)
+}}}
+
+(which provides a name and a type for a given index). FIXME: The (*oml_filter_meta) parameter takes precedence if both are provided, but may still inspect the OmlFilterDef. In an OmlFilterDef, the specific type OML_INPUT_VALUE can be used to indicate that the output schema for this field is the same as that of the input field. An example registration function could simply look like (and most of them actually are) as follows.
+{{{
+void
+omlf_register_filter_first (void)
+{
+  OmlFilterDef def [] =
+    {
+      { "first", OML_INPUT_VALUE },
+      { NULL, 0 }
+    };
+
+  omlf_register_filter ("first",
+            omlf_first_new,
+            NULL,
+            sample,
+            process,
+            newwindow,
+            meta,
+            def);
+}
+}}}
+The role of the
+{{{
+void* (*oml_filter_create) (OmlValueT type,  OmlValue* result)
+}}}
+
+function is to initialise a new InstanceData for the filter, to which it returns a pointer. This is then included by OML into a proper filter instance type, struct _omlFilter*, which is passed as f to the other method. The InstanceData is accessible from there as (InstanceData*)f->instance_data, and usually used as self. The type argument contains the data type of its input. FIXME: What if several inputs? The result argument is a pointer OML-initialised structure corresponding to the filter's output schema (as derived through the OmlFilterDef structure and the (*oml_filter_meta) function). The actual instance of the filter is really its InstanceData, corresponding to the structure defined in the header. The role of the instance-creation function is thus to allocate memory for a new such structure, and initialise it depending on the parameters, then return it to OML.
+
+XXX: design issue: why store OmlValue* result in the InstanceData rather than in the filter itself and access it as f->result? The skeleton of such a function typically is
+{{{
+void*
+omlf_EXAMPLE_new(
+  OmlValueT type,
+  OmlValue* result
+) {
+  int i;
+  InstanceData* self = (InstanceData *)malloc(sizeof(InstanceData));
+
+  if (self) {
+    memset(self, 0, sizeof(InstanceData));
+
+    self->result = result;
+    self->result[0].type = type; /* FIXME: What if there are several outputs of various types? */
+    /* Initialise other instance data here*/
+
+  } else {
+    logerror ("Could not allocate %d bytes for EXAMPLE filter instance data\n",
+        sizeof(InstanceData));
+    return NULL;
+  }
+
+  return self;
+}
+}}}
+FIXME: How can the
+{{{
+int (*oml_filter_set) (struct _omlFilter* filter, const char* name, OmlValue* value)
+}}}
+function be used?
+
+The measurement samples are reported to the filter via its
+
+{{{
+int (*oml_filter_input) (struct _omlFilter* filter, OmlValue* value)
+}}}
+
+The value can be copied verbatim (which the filter does), or accessed using such functions as oml_value_to_double() before being stored in, and depending on the content of, the InstanceData. A basic input function (which just copies the sample over its result - therefor creating some sort of a filter) would look as follow:
+{{{
+#include "oml_value.h" 
+
+/* ... */
+
+static int
+sample(
+    OmlFilter* f,
+    OmlValue * value  //! values of sample
+) {
+  InstanceData* self = (InstanceData*)f->instance_data;
+  OmlValueU* v = &value->value;
+  OmlValueT type = value->type;
+
+  if (type != self->result[0].type) {
+    logerror ("Different type from initial definition\n");
+    return 0;
+  }
+  return oml_value_copy(v, type, &self->result[0]);
+
+  return 0;
+}
+}}}
+
+When a filter output is required (after the required number of samples has been received, or the sampling time has elapsed), OML calls the
+
+{{{
+int (*oml_filter_output) (struct _omlFilter* filter, struct _omlWriter* writer)
+}}}
+
+function to have the filter output the filterred data using the provided writer. This is typically done by filling self->result (i.e. the OmlValue* obtained at the creation of the instance which has been stored in the InstanceData), then passing it to the writer as
+writer->out(writer, self->result, f->output_count);
+
+A more thorough skeleton for such a function would be
+{{{
+static int
+process(
+  OmlFilter* f,
+  OmlWriter*  writer //! Write results of filter to this function
+) {
+  InstanceData* self = (InstanceData*)f->instance_data;
+
+  writer->out(writer, self->result, f->output_count);
+
+  /* Prepare the instance for a new sampling period */
+
+  return 0;
+}
+}}}
+
+Note that all but the filter registration function are static, as they are only referenced directly by the registration function, and later only called using function pointers by OML.
+
+Refer to source:lib/client/filter/first_filter.c for a full implementation.
+
+=== Adding a unit test for the filter ===
+The filters' unit test is in test/lib/check_liboml2_filters.c add a test for your new filter there. Test at least creation and output (test_filter_FILTER_{create,output}). Follow the examples already in the file (the averaging filter is a good starting point).
+
+This is not optional.
+
+== Registering the filter ==
+=== From OML ===
+A call to the filter's function which encapsulates omlf_register_filter() is sufficient. It should be added to function register_builtin_filters in source:lib/client/filter/factory.c. As the fi lter's own registration function is not declared in any header included by this file, its prototype (typically void rf(void)) has to be repeated just before the definition of register_builtin_filters.
+
+For the new filter's code to be included in the build, references to its header and implementation files must be added to source:lib/client/Makefile.am, in variable liboml2_la_SOURCES.
+
+=== From an OML client ===
+
+== Compiling the filter outside of OML ==