Project

General

Profile

Bug #5899

diag_tables.php- Poor performance with large tables

Added by NOYB NOYB over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Web Interface
Target version:
Start date:
02/16/2016
Due date:
% Done:

0%

Estimated time:
Affected Version:
2.3
Affected Architecture:

Description

Displaying large tables such as bogonsv6 has very poor page loading performance. More than a minute. Clocked at ~75 seconds. All client CPU. Compared to ver. 2.2.6 loading in about ~10 seconds.

Diag Tables Poor Performance.jpg (72.8 KB) Diag Tables Poor Performance.jpg NOYB NOYB, 02/16/2016 10:18 PM

History

#1 Updated by Chris Buechler over 3 years ago

  • Subject changed from Diag Tables Poor Performance to Diag Tables poor performance with large tables
  • Category set to Web Interface
  • Status changed from New to Confirmed

what browser? A comparison in Chrome on Windows and Linux is around 11 seconds on 2.2.6, 25 seconds on 2.3. That's a small enough diff for a 66,000+ line table that it's no big deal.

But in IE 11, it sits and spins and chews up several hundred MB RAM. Killed it after waiting 3 minutes with no reply displayed. Firefox is similarly slow, hangs up Firefox and points to jquery as a slow script after ~2.5 minutes.

#2 Updated by Jim Thompson over 3 years ago

  • Assignee set to Steve Beaver

#3 Updated by NOYB NOYB over 3 years ago

Oh sorry. Guess I was a bit too light on info there. But it was bad enough on both Chrome and IE that I just neglected to include further details.

Chrome:
What I posted in the original opening ticket is regarding Chrome. ~75 seconds for 2.3 and ~10 seconds for 2.2.6. A factor of 7 is quite a difference. In your 11 vs. 25 seconds that is more than 2x factor. Don't think I would consider that to be small.

IE:
Like you IE is far worse. On 2.3 I waited 5 to 6 minutes and IE finally crashed. It just closed without warning. As though I had closed it. Though on 2.2.6 IE is fine at ~10 seconds just like 2.3.

Windows 8.1 Pro

#4 Updated by NOYB NOYB over 3 years ago

Though on 2.2.6 IE is fine at ~10 seconds just like 2.3.
Meant to say ... just like Chrome.

#5 Updated by Steve Beaver over 3 years ago

  • Subject changed from Diag Tables poor performance with large tables to diag_tables.php- Poor performance with large tables

#6 Updated by Steve Beaver over 3 years ago

  • Status changed from Confirmed to Feedback
  • Assignee changed from Steve Beaver to NOYB NOYB

I have tested this by selecting "bogons" and then clicking "Update". This produces a very long table that on my system loads in < 2 seconds with FF, Chrome and Safari.

Would you do the same test please? I would like to verify that my test is valid before pursuing this.

#7 Updated by Chris Buechler over 3 years ago

bogons isn't long enough, bogonsv6 is though. Just picking bogonsv6 from the drop down on diag_tables.php will show the issue.

#8 Updated by NOYB NOYB over 3 years ago

As Chris mentioned. The bogons list isn't large enough. Need to select bogonsv6 to see the issue.

#9 Updated by Steve Beaver over 3 years ago

Using Firefox bogons6 just locks up the browser for ever. Even removing all the table code and disabling javascript like this:

foreach ($entries as $entry):
print($entry . "<br />");
endforeach;

kills Firefox and takes for ever in Chrome. I guess 680,000 lines of text will do that.

Perhaps we need to do some server side paging?

#10 Updated by Jim Thompson over 3 years ago

  • Priority changed from Normal to Low

lowering priority. This isn't required functionality (it's part of the diag menu, and there is a lot of poorly considered code in that area), and while it's slow, it's not broken, assuming your browser doesn't run out of ram.

#11 Updated by NOYB NOYB over 3 years ago

Same table in ver. 2.2.6 isn't a problem in any of those browsers.

#12 Updated by Jim Thompson over 3 years ago

understood, but I still have to perform triage, or we won't ever get to a 2.3-RELEASE

#13 Updated by NOYB NOYB over 3 years ago

Jim Thompson wrote:

lowering priority. This isn't required functionality (it's part of the diag menu, and there is a lot of poorly considered code in that area), and while it's slow, it's not broken, assuming your browser doesn't run out of ram.

That's broken.
Ability to view tables is required functionality.

#14 Updated by NOYB NOYB over 3 years ago

Steve Beaver wrote:

Using Firefox bogons6 just locks up the browser for ever. Even removing all the table code and disabling javascript like this:

foreach ($entries as $entry):
print($entry . "<br />");
endforeach;

kills Firefox and takes for ever in Chrome. I guess 680,000 lines of text will do that.

Perhaps we need to do some server side paging?

Where you coming up with 680,000? The bogonsv6 is only 66,386 records.

Remove that line break and loading will be nearly instant. Second or two at most. Even with the javascript stuff still there.

This loads in a few seconds. Removing the tr and td tags and leaving only the pre tag it loads near instantly.

echo "Rows: " . count($entries) . "<br />";
echo "<tr><td><pre>";
foreach ($entries as $entry):
    print $entry . "\n";
endforeach;
echo "</pre></td></tr>";

#15 Updated by Steve Beaver over 3 years ago

OK so I was a factor of ten out :)

The slowness appears to be coming from the table sorting library. As a workaround I added a flag to prevent that from being loaded on this page. I will look for a better solution when times permits.

#16 Updated by NOYB NOYB over 3 years ago

Steve Beaver wrote:

OK so I was a factor of ten out :)

The slowness appears to be coming from the table sorting library. As a workaround I added a flag to prevent that from being loaded on this page. I will look for a better solution when times permits.

That doesn't seem to improve it here.
Removing the stylesheets links from head.inc lines 87,88,89 cuts the load time down to 15 seconds in Chrome.

I'm thinking the root of this is going to be something to do with the nested css.

#17 Updated by Chris Buechler over 3 years ago

still just locks up my Firefox and IE when attempting to load that page.

#18 Updated by NOYB NOYB over 3 years ago

Steve Beaver wrote:

Perhaps we need to do some server side paging?

Problem with that is losing the ability to search the table for an address. Not fond of that.

#19 Updated by NOYB NOYB over 3 years ago

Chris Buechler wrote:

still just locks up my Firefox and IE when attempting to load that page.

I'm displaying tables larger than 3000 records in a pre tag. Similar to what I showed above. Loads in just few seconds for both IE and Chrome. I can submit it if you like. But it really is just a Band-Aid. But at least it's functional and convenient to use.

#20 Updated by Steve Beaver over 3 years ago

Not loading the sorting stuff makes the difference between locking up the browser and slow but usable for me so I will retain that. I would welcome a PR that improves the speed.

In practice, is bogon6 the only case where we see tables of 60,000 entries? Other table types render a button so we can't just pre tag the whole table.

#21 Updated by NOYB NOYB over 3 years ago

There are a lots of lists available from many sources. Many of them are quite large. I use a CIPB (COUNTRY IP BLOCKS) U.S. networks list that is about 48K records.

Submitted my pre tag band-aid. PR # 2669.

#22 Updated by NOYB NOYB over 3 years ago

I'd prefer not to have the whole thing pre tagged. It's nice to have the tables formatted where performance is practical. Plus it doesn't blind you with a bright background in the dark theme. Though I'm sure that could be resolved with some css.

#23 Updated by Steve Beaver over 3 years ago

Agreed. Thanks again for the patch. I'll as Jared to look over the CSS to see if he can find an issue there.

#24 Updated by Steve Beaver over 3 years ago

  • Assignee changed from NOYB NOYB to Jared Dillard

#25 Updated by Steve Beaver over 3 years ago

Doing some more testing this morning I find that the browser lockup I was seeing is peculiar to one particular version of Firefox on OS-x (45.0b6). When using the developer version, Chrome or Safari NOYB's work-around is super fast, displaying the 66,000 entry table in less than 2 seconds. I will assume its a glitch in that version (which is a Beta version after all)

#26 Updated by Chris Buechler over 3 years ago

  • Status changed from Feedback to Resolved

much faster than 2.2.x in all browsers now

Also available in: Atom PDF