Profiler

A key issue in developing fast programs using the MonetDB database back-end requires a keen eye on where performance is lost. Although performance tracking and measurements are highly application dependent, a simple to use tool makes life a lot easier (See stethoscope).

Activation of the performance monitor has a global effect, i.e. all concurrent actions on the kernel are traced, but the events are only sent to the client who initiated the profiler thread.

The profiler event can be handled in several ways. The default strategy is to ship the event record immediately over a stream to a performance monitor. An alternative strategy is preparation of off-line performance analysis, where event information is retained in tables.

To reduce the interference of performance measurement with the experiments, the user can use an event cache, which is emptied explicitly upon need.

Profiler module

MODULE profiler;

UNSAFE PATTERN profiler.cleanup():void;
COMMENT "Remove the temporary tables for profiling";

PATTERN profiler.closestream():void;
COMMENT "Stop offline proviling";

COMMAND profiler.cpuload(X_0:lng, X_1:lng, X_2:lng, X_3:lng, X_4:lng) (X_5:int, X_6:int);
COMMENT "Calculate the average cpu load percentage and io waiting times";

COMMAND profiler.cpustats() (X_0:lng, X_1:lng, X_2:lng, X_3:lng, X_4:lng);
COMMENT "Extract cpu statistics from the kernel";

COMMAND profiler.getDiskReads():lng;
COMMENT "Obtain the number of physical reads";

COMMAND profiler.getDiskWrites():lng;
COMMENT "Obtain the number of physical reads";

COMMAND profiler.getSystemTime():lng;
COMMENT "Obtain the user timing information.";

PATTERN profiler.getTrace(X_0:str):bat[:any_1];
COMMENT "Get the trace details of a specific event";

COMMAND profiler.getUserTime():lng;
COMMENT "Obtain the user timing information.";

COMMAND profiler.getlimit():int;
COMMENT "Get profiler limit";

COMMAND profiler.noop():void;
COMMENT "Fetch any pending performance events";

PATTERN profiler.openstream():void;
COMMENT "Start profiling the events, send to output stream";

PATTERN profiler.openstream(X_0:int):void;
COMMENT "Start profiling the events, send to output stream";

UNSAFE COMMAND profiler.setheartbeat(X_0:int):void;
COMMENT "Set heart beat performance tracing";

UNSAFE COMMAND profiler.setlimit(X_0:int):void;
COMMENT "Set profiler limit";

UNSAFE PATTERN profiler.start():void;
COMMENT "Start offline performance profiling";

UNSAFE PATTERN profiler.starttrace():void;
COMMENT "Start collecting trace information";

UNSAFE PATTERN profiler.stop():void;
COMMENT "Stop offline performance profiling";

UNSAFE PATTERN profiler.stoptrace():void;
COMMENT "Stop collecting trace information";

Querylog module

MODULE querylog;

PATTERN querylog.append(X_0:str, X_1:str, X_2:str, X_3:timestamp):void;
COMMENT "Add a new query call to the query log";

PATTERN querylog.call(X_0:timestamp, X_1:timestamp, X_2:str, X_3:lng, X_4:lng, X_5:lng, X_6:int, X_7:int):void;
COMMENT "Add a new query call to the query log";

COMMAND querylog.context(X_0:str, X_1:str, X_2:str, X_3:str):void;
COMMENT "Noop operation, just marking the query";

COMMAND querylog.define(X_0:str, X_1:str, X_2:int):void;
COMMENT "Noop operation, just marking the query";

COMMAND querylog.disable():void;
COMMENT "Turn off the query logger";

COMMAND querylog.empty():void;
COMMENT "Clear the query log tables";

COMMAND querylog.enable():void;
COMMENT "Turn on the query logger";

COMMAND querylog.enable(X_0:int):void;
COMMENT "Turn on the query logger";

COMMAND querylog.isset():int;
COMMENT "Return status of query logger";

Sysmon module

MODULE sysmon;

UNSAFE PATTERN sysmon.pause(X_0:lng):void;
COMMENT "Suspend query execution with OID id";

UNSAFE PATTERN sysmon.pause(X_0:lng, X_1:str):void;
COMMENT "Sysadmin call, suspend query execution with OID id belonging to user";

PATTERN sysmon.queue() (X_0:bat[:lng], X_1:bat[:int], X_2:bat[:str], X_3:bat[:timestamp], X_4:bat[:str], X_5:bat[:str], X_6:bat[:timestamp], X_7:bat[:int], X_8:bat[:int]);
COMMENT "A queue of queries that are currently being executed or recently finished";

PATTERN sysmon.queue(X_0:str) (X_1:bat[:lng], X_2:bat[:int], X_3:bat[:str], X_4:bat[:timestamp], X_5:bat[:str], X_6:bat[:str], X_7:bat[:timestamp], X_8:bat[:int], X_9:bat[:int]);
COMMENT "Sysadmin call, to see either the global queue or user queue of queries that are currently being executed or recently finished";

UNSAFE PATTERN sysmon.resume(X_0:lng):void;
COMMENT "Resume query execution with OID id";

UNSAFE PATTERN sysmon.resume(X_0:lng, X_1:str):void;
COMMENT "Sysadmin call, resume query execution with OID id belonging to user";

UNSAFE PATTERN sysmon.stop(X_0:lng):void;
COMMENT "Stop query execution with OID id";

UNSAFE PATTERN sysmon.stop(X_0:lng, X_1:str):void;
COMMENT "Sysadmin call, stop query execution with OID id belonging to user";

PATTERN sysmon.user_statistics() (X_0:bat[:str], X_1:bat[:lng], X_2:bat[:lng], X_3:bat[:timestamp], X_4:bat[:timestamp], X_5:bat[:lng], X_6:bat[:str]);
COMMENT "";

Status module

MODULE status;