Bug #2935
closed
Revert "Backup RRD files using the xml dump and restore from RRD tools" changes
0%
Description
The changes for ticket #2123 work, but in the version of rrdtool we're stuck on for the time being, it consumes far too much memory to be a good idea. Going back to the previous method is adequate to resolve for now. When we get on a newer rrdtool we can re-apply the changes on ticket #2123.
The only downside is RRD data can't be restored cross-architecture, which has never worked. The current status, it won't work on systems that don't have 30-40 times as much memory as the size of the RRD files (which might be several MB).
Updated by Seth Mos over 11 years ago
Can we atleast have a look if we can optimize the way we handle those large memory arrays during restore? I'm doing the same thing on RRD upgrade with dumping and restoring to upgrade the fields, and it works there on a 128MB install now.
The trick is to nest the PHP functions to reduce the memory footprint.
Updated by Renato Botelho over 11 years ago
- Assignee changed from Renato Botelho to Seth Mos
Assign it to you Seth, since you mentioned at #2123 you want to take a look. If you figure out there is no way to fix it, just assign it back to me and I can revert necessary changes.
Updated by Chris Buechler over 11 years ago
Seth, can you get this done within at most a week, or else explain enough to Renato so he can pick up on it? We're going to release in the near future, need to have this addressed in a way that doesn't require vast amounts of RAM to restore RRD data. Though I guess really we need to test to see as of today how much RAM is really required for this to function.
Updated by Seth Mos over 11 years ago
I've created a 128MB ram VM and restored my working soekris config onto it, it succeeded. Granted this was a full install.
I've reinstalled without swap and it still succeeded in restoring the 4.8MB xml file with RRD data. All graphs were present after reboot too.
Maybe there is something else specific about the backup file that it triggers over. Not accidentally a full backup I hope.
Maybe I'm missing something obvious, I'll try a 96MB VM just to break things. Granted, I did restore a 2.1 config onto a 2.1. Could it have been a 2.0 config that it attempts to restore? Because in 2.0 we just included the base64 of the tgz file. In 2.1 that is a base64 of the XML data. Which is something very different.
My guess is that restoring a 2.0 config onto a 2.1 install can trigger this. I suppose we can just check the version number of the XML file and deduce if we need to restore the tgz instead, the reboot should take care of upgrading the files. I suppose that this is the missing upgrade code.
Updated by Seth Mos over 11 years ago
I just pushed another 5+ MB config (from 2.1) with 6 interfaces and assorted RRD files into a current 2.1 install and it appears to load all RRD files fine. This was with 128MB ram.
I also loaded the 2.0 config with RRD files I had into 2.1 and that converted without issue too. This on a VM with 128MB without swap.
I've confirmed that a relatively small 2.1 config with IPv6 won't succesfully boot at a 100MB ram.
Updated by Seth Mos over 11 years ago
- Status changed from New to Feedback
I've committed a different RRD upgrade scheme, please test. WorksForMe(tm)
https://redmine.pfsense.org/projects/pfsense/repository/revisions/fcaa56b19aa3f38c6e05bec5d3e33a6042734eac
Updated by Chris Buechler about 11 years ago
- Status changed from Feedback to Resolved