Project

General

Profile

Actions

Bug #1758

closed

Upgrade fails to upgrade RRD data for traffic and packets

Added by Eddie Atherton over 12 years ago. Updated over 12 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Upgrade
Target version:
Start date:
08/05/2011
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
2.0
Affected Architecture:
i386

Description

Running a nanobsd upgrade from 1.2.3 to 2.0-RC3 gives the following errors:

Aug 4 22:22:20 roadblock php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/lan-traffic.rrd N:U:U:U:U' returned exit code '1', the output was 'ERROR: expected 2 data source readings (got 4) from N:U:U:U:U'
Aug 4 22:22:20 roadblock php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/lan-packets.rrd N:U:U:U:U' returned exit code '1', the output was 'ERROR: expected 2 data source readings (got 4) from N:U:U:U:U'
Aug 4 22:22:20 roadblock php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/wan-traffic.rrd N:U:U:U:U' returned exit code '1', the output was 'ERROR: expected 2 data source readings (got 4) from N:U:U:U:U'
Aug 4 22:22:20 roadblock php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/wan-packets.rrd N:U:U:U:U' returned exit code '1', the output was 'ERROR: expected 2 data source readings (got 4) from N:U:U:U:U'
Aug 4 22:22:20 roadblock php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/ipsec-traffic.rrd N:U:U:U:U' returned exit code '1', the output was 'ERROR: expected 2 data source readings (got 4) from N:U:U:U:U'
Aug 4 22:22:20 roadblock php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/ipsec-packets.rrd N:U:U:U:U' returned exit code '1', the output was 'ERROR: expected 2 data source readings (got 4) from N:U:U:U:U'

Following this, the graphs for traffic and packets is not available.

Cheers.

Actions #1

Updated by Seth Mos over 12 years ago

Caution, if the config is Upgraded on nanobsd platforms the converted RRD files are not immediately saved to the flash. This means that if the box is powercycled instead of rebooted after a conversion the converted RRD files are lost.

Since the conversion takes a really really long time on a Alix or similar the device is often power cycled while it assumed it hangs (which it isn't). This causes a upgraded config to be saved, so no other attempt will be made. This still leaves you with the old RRD files.

Actions #2

Updated by Eddie Atherton over 12 years ago

These errors showed up on the upgrade, and are associated with the upgrade process, not trying to display the graphs.

Also, during my tests, the box was never powercycled. It was always shut down, or rebooted, using the GUI.

Cheers.

Actions #3

Updated by Seth Mos over 12 years ago

That's good to know, we'll investigate. We're still not quite sure what's happening, some ipv6 builds seem to hit it more.

Actions #4

Updated by Chris Buechler over 12 years ago

  • Target version changed from 2.0 to 2.0.1
Actions #5

Updated by Eddie Atherton over 12 years ago

OK, I'm no expert on RRD, but have done a little more digging on this.

In 1.2.3, the Traffic and Packet RRDs were populated with 2 values, Ibytes, and Obytes, into 2 data points: "in" and "out".

Now, under 2.0, it appears that you are trying to collect 4 values, I4Pass, I4Block, O4Pass, and O4Block, into 4 data points: "inpass", "outpass", "inblock", and "outblock".

However, the RRD, in 2.0, is the "old" 1.2.3 one, hence the errors above, about trying to set 4 values, when only 2 are expected.

This would also explain the errors trying to display the graph:

Oct 9 22:46:02 pfHacom php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: No DS called 'inpass' in '/var/db/rrd/wan-traffic.rrd'/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/wan-traffic.rrd-8hour.png --start 1318196762 --end 1318225562 --vertical-label "bits/sec" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "`hostname` - WAN :: Traffic - 8 hours - 1 minute average" --height 200 --width 620 DEF:wan-in_bytes_pass=/var/db/rrd/wan-traffic.rrd:inpass:AVERAGE DEF:wan-out_bytes_pass=/var/db/rrd/wan-traffic.rrd:outpass:AVERAGE DEF:wan-in_bytes_block=/var/db/rrd/wan-traffic.rrd:inblock:AVERAGE DEF:wan-out_bytes_block=/var/db/rrd/wan-traffic.rrd:outblock:AVERAGE CDEF:"wan-in_bits_pass=wan-in_bytes_pass,8,*" CDEF:"wan-out_bits_pass=wan-out_bytes_pass,8,*" CDEF:"wan-in_bits_block=wan-in_bytes_block,8,*" CDEF:"wan-out_bits_block=wan-out_bytes_block,8,*" CDEF:"wan-in_bytes=wan-in_bytes_pass,wan-in_bytes_block,+" CDEF:"wan-out_bytes=wan-out_bytes_pass,wan-out_bytes_bloc

Where the command is looking for the new data points: "inpass", "outpass", "inblock", and "outblock".

So, obviously, to upgrade from 1.2.3 -> 2.0, the traffic and packet RRDs must be updated to the "new" format.

Now, what I don't know, is if there is supposed to be a step in the upgrade, to perform the change or not. If there is, then somehow it was skipped.

If not, then it needs to be in place, for the 1.2.3 RRD to be carried forward to 2.0.

Cheers,
Eddie

Actions #6

Updated by Seth Mos over 12 years ago

In config step x we run the following commands. The only things I can think of is that these have failed.

function upgrade_054_to_055() {
global $config;
global $g;

<snip>

mwexec("$rrdtool tune {$rrddbpath}/{$database} -r in:inpass 2>&1");
mwexec("$rrdtool tune {$rrddbpath}/{$database} -r out:outpass 2>71");

That command renames a datasource in a RRD archive from in to inpass and from out to outpass.
We then create a new empty rrd file, dump the xml from that and then layer the old data over the new rrd so that we fill it with data.

It's a rather cumbersome procedure that can take many minutes on a low power alix.

If you can send the rd databases from 1.2.3 my way I can try a lab upgrade.

Actions #7

Updated by Seth Mos over 12 years ago

Just a FYI in case people want to upgrade their 1.2.3 databases on their 2.0 installs. Executing the following code on the command prompt page will call the required function.

Do not run this on already upgraded databases as that can and will cause data loss. Make sure to download the RRD databases before attempting this.
@
include("upgrade_config.inc");
include("rrd.inc");
include("shaper.inc");

upgrade_054_to_055();@

Actions #8

Updated by Seth Mos over 12 years ago

  • Status changed from New to Feedback

Ok, so the code now triggers properly on a nanobsd system but it causes a Out of Memory situation on a 128MB system (tested in a VM).

I've checked in another patch that I hope prevents the OOM condition on upgrade as this would leave a system dead in the water on upgrade.

I've just tested the upgrade and it works on a 128MB system without triggering a OOM. What is strange is that the graphs work as intended but the 3 month and 1 year graphs is empty. Upto and including the 1 month graph is ok.

The 4 year graph shows data again but it's shown from the left and not the right.

Actions #9

Updated by Chris Buechler over 12 years ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF