pfSense

Jim Pingle wrote in #note-1:

That is expected. Use a URL table alias for large lists.

I'm not sure how this is expected. pfSense let me put a bunch of subnets and now doesn't allow to edit them. This seems like an obvious bug to me.

As to URL table, that requires me to upload those URLs somewhere first, which is really annoying. Basically to write a microservice that would take the file, split it into multiple and serve different chunks over different URLs. This is clearly something pfSense should handle transparently internally rather than having a hardcoded limit of 3000.

Trying to manage that many entries in the configuration isn't viable, not only does it consume too many resources in PHP, but it would also be impractical to store in config.xml and on top of that, browsers have difficulty rendering forms with that many controls. Paginating is a different difficult problem entirely. We've looked into it over the years and it would take considerable effort and given the other methods we have that DO work, it's not worth that effort to implement natively.

You worked around the safety controls that prevent it from breaking, so it's no surprise it doesn't function.

URL Table aliases exist to work around these limitations. Managing tens of thousands of entries in the GUI is not practical nor feasible.

You can drop the list in a file in /usr/local/www/ on the firewall and load them into a URL table alias from https://127.0.0.1/mylist.txt if nothing else.

Or consider installing the pfBlockerNG package which has different methods to manage large lists.

Jim Pingle wrote in #note-3:

Trying to manage that many entries in the configuration isn't viable, not only does it consume too many resources in PHP, but it would also be impractical to store in config.xml and on top of that, browsers have difficulty rendering forms with that many controls. Paginating is a different difficult problem entirely. We've looked into it over the years and it would take considerable effort and given the other methods we have that DO work, it's not worth that effort to implement natively.

You worked around the safety controls that prevent it from breaking, so it's no surprise it doesn't function.

URL Table aliases exist to work around these limitations. Managing tens of thousands of entries in the GUI is not practical nor feasible.

You can drop the list in a file in /usr/local/www/ on the firewall and load them into a URL table alias from https://127.0.0.1/mylist.txt if nothing else.

Or consider installing the pfBlockerNG package which has different methods to manage large lists.

That is a bit better, but since it is so simple to implement, why wouldn't pfSense under the hood slice big URL into multiple and apply them individually or store big lists in files instead of individual records?
It feels like an arbitrary limitation and that there should be a simpler approach.