Thursday, September 9, 2010

Updating Metric Collection Schedules

One of the primary features of RHQ is monitoring. Metrics can be collected for resources across the inventory, that metric data is aggregated, and then available for viewing in graphs and tables. Measurements are collected at scheduled intervals. These collection schedules are configurable on a per-resource or per-group basis. Collection intervals can be increased or decreased, and metric collections can be turned on or off as well. There might be any number of reasons why you might want to adjust metric collection schedules. One reason might be that collections are occurring too frequently and in turn causing performance degradation on the host machine.

RHQ exposes APIs for updating measurement schedules through the CLI. I find that some of the APIs exposed through the CLI are not the most intuitive or require a thorough understanding of the RHQ domain classes and APIs. Some of the APIs for dealing with measurements fall into this category. I have started working on putting together some utility scripts that can serve as higher-level building blocks for CLI scripts. My aim is to simplify various, common tasks when and where possible. I put together a JavaScript class that offers a more data-driven approach for updating measurement schedules. Let's look at an example.


The first thing we do is create an instance of MeasurementModule. This class expose properties and methods for working with measurements, most notably the method updateSchedules. We call this method on line 15. updateSchedules takes a single object which specifies the measurement schedule changes. That object is defined on lines 5 - 13. It has to define three properties - context, id, and schedules.

context accepts only two values, 'Resource' or 'Group'. This property declares whether the update is for an individual resource or for a resource group.

The value of context determines how id is interpreted. It refers either to a resource id or to a resource group id.

schedules is essentially a map that declares which schedules are to be updated. The keys are the measurement display names as you see them in the RHQ UI. The values can be one of three things. It can be the strings 'enabled' or 'disabled' indicating that that measurement collection should be enabled or disabled respectively. Or the value can be the collection interval specified as an integer. The collection interval is stored in milliseconds. Most collection intervals are on the order of minutes though.

It is a lot easier to read and write 30 minutes as instead of 1800000 milliseconds. To address this, MeasurementModule exposes the interval method to which we declare a reference on line 2 to facilitate in calculating the interval in a more readable way. We use MeasurementModule.time in conjunction with this method. time has three properties - seconds, minutes, hours. We see these in use on lines 9 and 11.

I think (hope) MeausurementModule offers a fairly straightforward approach for updating measurement schedules. It should allow you to make to make programmatic updates without having an in-depth understanding of the underling APIs.

There is at least one additional enhancement that I already intend to make. I want to provide a way to specify an update that applies to multiple schedules. Maybe something along these lines,


In this example we are updating schedules for a compatible group. We specify a collection interval of one hour for Measurement A, disable Measurement B, and all of the rest of the measurements are set to a collection interval of 30 minutes. I see this as a useful feature and will write another post when I have it implemented.

There is one last annoying detail that needs to be discussed before you start using MeausurementModule. The class uses some of the functions described in Utility Functions for RHQ CLI. Those functions are defined in another source file. When using the CLI in interactive mode, you can use the exec command to execute a script. That command (or an equivalent method/function) however is not available in non-interactive mode. I filed a bug for this a little while back. You can track the progress here if you are interested. This means that for now you need to run in interactive mode. Let's walk through a final example tying all of this together. Assume we are have already logged in through the CLI.

rhqadmin@localhost:7080$ exec -f path/to/util.js
rhqadmin@localhost:7080$ exec -f path/to/measurement_utils.js
rhqadmin@localhost:7080$ measurementModule = new MeaurementModule()
rhqadmin@localhost:7080$ // do some stuff with measurementModule

MeausrementModule is defined in the file measurement_utils.js. A few scripts including measurement_utils.js are shiped in the latest release of RHQ which can be downloaded from here. They are packaged in the CLI in the samples directory.

1 comment:

  1. John,

    Really useful post however it looks like the utility classes have been removed from the linked location.

    Mark

    ReplyDelete