sirius.web.crunchlog

Class Crunchlog

java.lang.Object

sirius.web.crunchlog.Crunchlog

public class Crunchlog
extends Object

Provides a facility to write large amounts of semi structured data into log files to be queried later.

The Crunchlog manages a directory stucture where all submitted records are stored in compressed files. These contain a JSON object per line which represents the map that was submitted. Being a map, records with varying attributes can be stored. The crunchlog will start a file per day, and limit the file size to a sane value (10 MB). Being JSON files, processing of the generated files is quite easy. The files are compressed as JSON contains a lot of reoccuring elements. Also many attribute names will be used frequently.

The implementation of the Crunchlog is put into CrunchlogKernel as some frameworks which require public methods are utilized.

The Crunchlog contains some safety switches to disable itself if either no valid base directory is present or if the underlying disk runs out of free space.

Note that crunchlog files older than one year are deleted by DeleteOldCrunchlogs. If files need to be kept longer, collectAllCompletedFiles(Consumer) can be invoked to collect all available files and store them in a permanent location. The given consumer may also delete the files by itself, to free up disk space.

To disable the crunchlog entirely, set frameworks.crunchlog = false in the system config.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	Crunchlog.LogBuilder Creates a new builder for a log entry in the crunchlog.

Field Summary

Fields

Modifier and Type	Field and Description
static String	FIELD_DATE This field will be automatically added to each record, containing the timestamp when is was logged.
static String	FIELD_TYPE It is highly recommended to add a value for this field in each record to better support filtering for similar events.
protected static sirius.kernel.health.Log	LOG

Constructor Summary

Constructors

Constructor and Description
Crunchlog()

Method Summary

Modifier and Type	Method and Description
void	collectAllCompletedFiles(Consumer<File> fileProcessor) Iterates over all completely written files (excluding the one currently being written to).
Crunchlog.LogBuilder	log(String event) Creates a new builder to create a log message using fluid method calls.
void	safeLog(String event, sirius.kernel.commons.Callback<Crunchlog.LogBuilder> log) Implements a safe way of logging to the crunchlog.
void	submit(sirius.kernel.commons.Context data) Submits a record to the crunchlog which will be eventually persisted to disk.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final sirius.kernel.health.Log LOG

FIELD_DATE

public static final String FIELD_DATE

This field will be automatically added to each record, containing the timestamp when is was logged.

See Also:

Constant Field Values

FIELD_TYPE

public static final String FIELD_TYPE

It is highly recommended to add a value for this field in each record to better support filtering for similar events.

See Also:

Constant Field Values

Constructor Detail

Crunchlog

public Crunchlog()

Method Detail

log

@CheckReturnValue
public Crunchlog.LogBuilder log(String event)

Creates a new builder to create a log message using fluid method calls.

Consider using safeLog(String, Callback) to perform logging in a critical section.

Parameters:

event - the event name to log

Returns:

a builder to create and submit a log message

safeLog

public void safeLog(String event,
                    sirius.kernel.commons.Callback<Crunchlog.LogBuilder> log)

Implements a safe way of logging to the crunchlog.

As the crunchlog might be involved in critical operations, where a failure during logging must to abort the actual operation, this method creates (and submits) a Crunchlog.LogBuilder while surrounding all calls with a try / catch block, which logs all errors, but does not throw an exception.

Parameters:

event - the event to log

log - the consumer to actually apply the logging

submit

public void submit(sirius.kernel.commons.Context data)

Submits a record to the crunchlog which will be eventually persisted to disk.

Parameters:

data - the record to persist

collectAllCompletedFiles

public void collectAllCompletedFiles(Consumer<File> fileProcessor)

Iterates over all completely written files (excluding the one currently being written to).

This could be used to upload all files to another repository for further processing. The consumer may delete the files to better manage disk usage.

Parameters:

fileProcessor - a consumer which will be supplied with all completely written files of the Crunchlog