https://redmine.pfsense.org/https://redmine.pfsense.org/favicon.ico?16780521162011-08-24T03:40:43ZpfSense bugtrackerpfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=68162011-08-24T03:40:43ZSeth Mosseth.mos@dds.nl
<ul></ul><p>Caution, if the config is Upgraded on nanobsd platforms the converted RRD files are not immediately saved to the flash. This means that if the box is powercycled instead of rebooted after a conversion the converted RRD files are lost.</p>
<p>Since the conversion takes a really <em>really</em> long time on a Alix or similar the device is often power cycled while it assumed it hangs (which it isn't). This causes a upgraded config to be saved, so no other attempt will be made. This still leaves you with the old RRD files.</p> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=69052011-09-04T16:26:46ZEddie Athertonstunnel@attglobal.net
<ul></ul><p>These errors showed up on the upgrade, and are associated with the upgrade process, not trying to display the graphs.</p>
<p>Also, during my tests, the box was never powercycled. It was always shut down, or rebooted, using the GUI.</p>
<p>Cheers.</p> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=69062011-09-04T16:29:05ZSeth Mosseth.mos@dds.nl
<ul></ul><p>That's good to know, we'll investigate. We're still not quite sure what's happening, some ipv6 builds seem to hit it more.</p> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=70402011-09-16T23:58:47ZChris Buechlercbuechler@gmail.com
<ul><li><strong>Target version</strong> changed from <i>2.0</i> to <i>2.0.1</i></li></ul> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=71682011-10-10T13:40:54ZEddie Athertonstunnel@attglobal.net
<ul></ul><p>OK, I'm no expert on RRD, but have done a little more digging on this.</p>
<p>In 1.2.3, the Traffic and Packet RRDs were populated with 2 values, Ibytes, and Obytes, into 2 data points: "in" and "out".</p>
<p>Now, under 2.0, it appears that you are trying to collect 4 values, I4Pass, I4Block, O4Pass, and O4Block, into 4 data points: "inpass", "outpass", "inblock", and "outblock".</p>
<p>However, the RRD, in 2.0, is the "old" 1.2.3 one, hence the errors above, about trying to set 4 values, when only 2 are expected.</p>
<p>This would also explain the errors trying to display the graph:</p>
<p>Oct 9 22:46:02 pfHacom php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: No DS called 'inpass' in '/var/db/rrd/wan-traffic.rrd'/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/wan-traffic.rrd-8hour.png --start 1318196762 --end 1318225562 --vertical-label "bits/sec" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "`hostname` - WAN :: Traffic - 8 hours - 1 minute average" --height 200 --width 620 DEF:wan-in_bytes_pass=/var/db/rrd/wan-traffic.rrd:inpass:AVERAGE DEF:wan-out_bytes_pass=/var/db/rrd/wan-traffic.rrd:outpass:AVERAGE DEF:wan-in_bytes_block=/var/db/rrd/wan-traffic.rrd:inblock:AVERAGE DEF:wan-out_bytes_block=/var/db/rrd/wan-traffic.rrd:outblock:AVERAGE CDEF:"wan-in_bits_pass=wan-in_bytes_pass,8,*" CDEF:"wan-out_bits_pass=wan-out_bytes_pass,8,*" CDEF:"wan-in_bits_block=wan-in_bytes_block,8,*" CDEF:"wan-out_bits_block=wan-out_bytes_block,8,*" CDEF:"wan-in_bytes=wan-in_bytes_pass,wan-in_bytes_block,+" CDEF:"wan-out_bytes=wan-out_bytes_pass,wan-out_bytes_bloc</p>
<p>Where the command is looking for the new data points: "inpass", "outpass", "inblock", and "outblock".</p>
<p>So, obviously, to upgrade from 1.2.3 -> 2.0, the traffic and packet RRDs must be updated to the "new" format.</p>
<p>Now, what I don't know, is if there is supposed to be a step in the upgrade, to perform the change or not. If there is, then somehow it was skipped.</p>
<p>If not, then it needs to be in place, for the 1.2.3 RRD to be carried forward to 2.0.</p>
<p>Cheers,<br />Eddie</p> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=71772011-10-10T15:39:24ZSeth Mosseth.mos@dds.nl
<ul></ul><p>In config step x we run the following commands. The only things I can think of is that these have failed.</p>
<p>function upgrade_054_to_055() {<br /> global $config;<br /> global $g;</p>
<p><snip></p>
<p>mwexec("$rrdtool tune {$rrddbpath}/{$database} -r in:inpass 2>&1");<br />mwexec("$rrdtool tune {$rrddbpath}/{$database} -r out:outpass 2>71");</p>
<p>That command renames a datasource in a RRD archive from in to inpass and from out to outpass.<br />We then create a new empty rrd file, dump the xml from that and then layer the old data over the new rrd so that we fill it with data.</p>
<p>It's a rather cumbersome procedure that can take many minutes on a low power alix.</p>
<p>If you can send the rd databases from 1.2.3 my way I can try a lab upgrade.</p> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=71822011-10-13T06:00:50ZSeth Mosseth.mos@dds.nl
<ul></ul><p>Just a FYI in case people want to upgrade their 1.2.3 databases on their 2.0 installs. Executing the following code on the command prompt page will call the required function.</p>
<p>Do not run this on already upgraded databases as that can and will cause data loss. Make sure to download the RRD databases before attempting this.<br />@<br />include("upgrade_config.inc");<br />include("rrd.inc");<br />include("shaper.inc");</p>
<p>upgrade_054_to_055();@</p> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=72122011-10-18T05:54:29ZSeth Mosseth.mos@dds.nl
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Feedback</i></li></ul><p>Ok, so the code now triggers properly on a nanobsd system but it causes a Out of Memory situation on a 128MB system (tested in a VM).</p>
<p>I've checked in another patch that I hope prevents the OOM condition on upgrade as this would leave a system dead in the water on upgrade.</p>
<p>I've just tested the upgrade and it works on a 128MB system without triggering a OOM. What is strange is that the graphs work as intended but the 3 month and 1 year graphs is empty. Upto and including the 1 month graph is ok.</p>
<p>The 4 year graph shows data again but it's shown from the left and not the right.</p> pfSense - Bug #1758: Upgrade fails to upgrade RRD data for traffic and packets https://redmine.pfsense.org/issues/1758?journal_id=74312011-12-06T05:00:14ZChris Buechlercbuechler@gmail.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul>