Trend clean

[jmir]

This example will change all valid trend values by x2. Recommended way of testing is to create a new trend log with the same parameters as the source.

Code:

infile = '/tmp/trends/t1.trend'
outfile = '/tmp/trends/t7.trend'

function adjust(value)
  value = value * 2
  return value
end

ffi = require('ffi')

function stringtodouble(chunk)
  local dbl = ffi.new('double[1]')
  ffi.copy(dbl, chunk, 8)
  return dbl[ 0 ]
end

function doubletostring(val)
  local dbl = ffi.new('double[1]')
  dbl[ 0 ] = val
  return ffi.string(dbl, 8)
end

info = require('rrd').info(infile)

data = io.readfile(infile)
offset = info.header_size

buf = { data:sub(1, offset) }

while true do
  chunk = data:sub(offset + 1, offset + 8)

  if #chunk == 8 then
    value = stringtodouble(chunk)

    -- skip NaN (empty/undefined value)
    if value == value then
      value = adjust(value)
    end

    buf[ #buf + 1 ] = doubletostring(value)
  else
    break
  end

  offset = offset + 8
end

data = table.concat(buf)
io.writefile(outfile, data)

For counter type the editing is trickier. The stored value is not the actual counter value but a per-second change rate. This example will log the maximum of all stored values without writing to a different trend log file. You can use this value for replacement in the previous script.

Code:

infile = '/tmp/trends/t1.trend'

max = 0

function check(value)
  max = math.max(max, value)
end

ffi = require('ffi')

function stringtodouble(chunk)
  local dbl = ffi.new('double[1]')
  ffi.copy(dbl, chunk, 8)
  return dbl[ 0 ]
end

info = require('rrd').info(infile)

data = io.readfile(infile)
offset = info.header_size

while true do
  chunk = data:sub(offset + 1, offset + 8)

  if #chunk == 8 then
    value = stringtodouble(chunk)

    -- skip NaN (empty/undefined value)
    if value == value then
      check(value)
    end
  else
    break
  end

  offset = offset + 8
end

log(max)

Note that when replacing values you should not use direct comparison (==) for floating point values due to possible rounding errors.

Hi, 
I've used this scripts to replace erroneus values (due to power losses...) on counter type trends, but I've seen that daily grouped values are still wrong.
Are this values calculated from hourly values? Or are stored? Is there any way to update them?