pfSense

adding comments from duplicate #2664

After upgrading from 1.2.3 to 2.1 the rrd traffic & packets graphs no longer work. However graphs are OK if upgrading to 2.0.1

Further investigation shows that upgrade_080_to_081 is failing to update these graphs.
From the system.log:
Oct 24 15:31:55 pfsense php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool restore -f /tmp/lan-traffic.rrd.new.xml /var/db/rrd//lan-traffic.r
rd 2>&1' returned exit code '1', the output was 'ERROR: No <v> tag found'

Examination of /tmp/lan-traffic.rrd.new.xml shows that conversion of one of the databases is incomplete.
<row>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
<v>0.0000000000e+00 </v>
<v>0.0000000000e+00 </v>
<v>0.0000000000e+00 </v>
<v>0.0000000000e+00 </v>
</row>
<row>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
<v>0.0000000000e+00 </v>
<v>0.0000000000e+00 </v>
<v>0.0000000000e+00 </v>
<v>0.0000000000e+00 </v>
</row>
<row>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
</row>
<row>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
<v> NaN </v>
</row>
I added tracing to migrate_rrd_format in /etc/inc/rrd.inc:
$rowcountold = count($rrdoldxml['rra'][$i]['database']['row']);
$rowcountnew = count($rrdnewxml['rra'][$i]['database']['row']);
$rowcountdiff = $rowcountnew - $rowcountold;
echo "rowcountdiff $rowcountdiff\n";
and repeated the upgrade manually. This resulted in repeated output of:
rowcountdiff 0
rowcountdiff 0
rowcountdiff -2000
rowcountdiff 0
so it appears that one of the old databases is larger that the new one and this is not handled by the following script.

Changing:

$rrdcreate .= "RRA:AVERAGE:0.5:60:1000 ";
to
$rrdcreate .= "RRA:AVERAGE:0.5:60:3000 ";

in upgrade_080_to_081 allows the upgrade to complete and the graphs are now visible. However I am not clear regarding possible, unexpected, side-effects of this change. Is this OK for a fix?

After further consideration I think it would be better to reduce the number of rows to match the 2.1 definition.
Also it appears that the first 2000 rows of the offending databases are all NaN, so no information would be lost.
Attached is a patch for rrd.inc to remove the excess rows.
With this patch applied the rrd upgrade to 2.1 completes and all graphs appear to be correct.

File paths updated to ease patch application. Patch contents unchanged.

There was a bug on functions that convert rrd databases to new format during upgrade. It was fixed and this fix should help this case.

Applied in changeset 84683e42c96455d90b4cec5545bdedb5071ac5f1.

thanks