Project

General

Profile

Actions

Todo #596

closed

Varnish package suggestions for VCL syntax checking

Added by Scott Ullrich almost 14 years ago. Updated over 9 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
05/13/2010
Due date:
% Done:

0%

Estimated time:
Plus Target Version:

Description

I work in a web shop design company and we use varnish as a reverse proxy for mostly sites builded using Drupal, Codeignator, joomla etc, our biggest site is about of 30 million visits per month and is using varnish.

Our varnish config is pretty custom to the site and we normally use varnishadm to load changes in the configuration or when we rollback config files and we have a custom module in drupal to PURGE cache from varnish using port 6082 ( default for varnish admin console).

So base on that explanation I will like to know if is possible to implement the following :

vlc.load and vlc.use = With varnishadm you can compile a .vlc file on the fly without changing the actual running config, this make thing easy when you want to check the syntax and when you want to change the configuration of a running varish instance on the fly without restarting the process, vlc.load load the compiles and load the config in memory and vlc.use uses the configuration after it's compiled. Will be useful to have it on the varnish module so you can have multiples configs in memory that you can use to test or rollback without restarting the process.

Config templates and backups = Will be really cool to have templates of basic typical configurations and the ability to upload new ones, what I mean with templates is configuration files standard for some frameworks or your own sites, sometimes you can have aggressive caching in varnish and more passive caching configuration that you want to test, so you can have different config to chose in case that you need and again this could be tight to the ability of using. varnishadm

Purge tab = Sometimes in varnish is really difficult to find out if a item or object has been cache by varnish if you don't know to much about headers because there is no way to query the cache db that varnish uses, so if there is a tool that allow to clean up all the cache or a specific object in the cache base on regex or hostname or url will be really handy, same as varnishadm -T localhost:6082 purge.url "^/images/".

Varnishtop Tab = Varnishtop is really handy to check connections from specific source or the most popular url or any other information that varnishtop can give that it could be use to be able to tune up your caching options better so you can improve your configuration.

Varnishhits Tab = shows the live hit and misses will be handy to have in a tab with a refresh interval.

Advance Startup options = Sometimes depending on the application that you have or the amount of users you need to set up some startup options to varnish to avoid 503 errors or such, this options are passed has parameters to the varnishd command, will be really handy to be able to specify the startup options.

Multiple varnish server definition = When you have two or more varnish servers running serving the same web app and you want to purge some content from the cache you will have to have some sort of multiple server definition and that has to be tight to the purge tab because if you purge a object from the cache then if you have two pfSenses running varnish then you will have different caches and that will be bad.

Varnish acl = for the Purge tab you have to create acl on varnish and that is not part of vcl_recive or fetch , error or etc, so that text box should be added to the custom config tab.

Varnish log Tab = Varnish log is really powerful for debugging and solve problem or bugs, will be really useful to have that on a tab like the varnishtop tab that you can refresh every n seconds and with filter options for something like $ varnishlog -c -o ReqStart 192.0.2.123 and maybe have a drop box with all the -o options supported by varnish.

If any if this options will be added then pfSense will have be careful to add firewall rules to permit the varnishadm port and maybe add the ability to change that port of varnish adm.

This is what it comes to my mind for now, I'm pretty sure that I will come up with something else that it could be useful for everyone that uses varnish on pfSense 2.0.

Actions #1

Updated by Marcello Silva Coutinho over 12 years ago

I work in a web shop design company and we use varnish as a reverse proxy for mostly sites builded using Drupal, Codeignator, joomla etc, our biggest site is about of 30 million visits per month and is using varnish.

Our varnish config is pretty custom to the site and we normally use varnishadm to load changes in the configuration or when we rollback config files and we have a custom module in drupal to PURGE cache from varnish using port 6082 ( default for varnish admin console).

So base on that explanation I will like to know if is possible to implement the following :

vlc.load and vlc.use = With varnishadm you can compile a .vlc file on the fly without changing the actual running config, this make thing easy when you want to check the syntax and when you want to change the configuration of a running varish instance on the fly without restarting the process, vlc.load load the compiles and load the config in memory and vlc.use uses the configuration after it's compiled. Will be useful to have it on the varnish module so you can have multiples configs in memory that you can use to test or rollback without restarting the process.

can't you do this at managmente interface?

Config templates and backups = Will be really cool to have templates of basic typical configurations and the ability to upload new ones, what I mean with templates is configuration files standard for some frameworks or your own sites, sometimes you can have aggressive caching in varnish and more passive caching configuration that you want to test, so you can have different config to chose in case that you need and again this could be tight to the ability of using. varnishadm

varnish settings has some 'easy to use' config options, but you can set custom atributes at load balancer to be include in host/url checks

Purge tab = Sometimes in varnish is really difficult to find out if a item or object has been cache by varnish if you don't know to much about headers because there is no way to query the cache db that varnish uses, so if there is a tool that allow to clean up all the cache or a specific object in the cache base on regex or hostname or url will be really handy, same as varnishadm -T localhost:6082 purge.url "^/images/".

you cant configure managment interface at varnish_settings tab

Varnishtop Tab = Varnishtop is really handy to check connections from specific source or the most popular url or any other information that varnishtop can give that it could be use to be able to tune up your caching options better so you can improve your configuration.

Varnishhits Tab = shows the live hit and misses will be handy to have in a tab with a refresh interval.

it's dificult to dump Varnishhits into a file or a json query

Advance Startup options = Sometimes depending on the application that you have or the amount of users you need to set up some startup options to varnish to avoid 503 errors or such, this options are passed has parameters to the varnishd command, will be really handy to be able to specify the startup options.

Implemented

Multiple varnish server definition = When you have two or more varnish servers running serving the same web app and you want to purge some content from the cache you will have to have some sort of multiple server definition and that has to be tight to the purge tab because if you purge a object from the cache then if you have two pfSenses running varnish then you will have different caches and that will be bad.

Implemented -> create a second configuration file, save it on pfsense box and put second daemon options in advanced startup.
Varnish requires that when using multiple daemons, you need to name them.
Custom startup options example:
-n apache
#set name on default daemon

#startup second daemon
/usr/local/sbin/varnishd \
-a :3129 \
-n squid \
-f /var/etc/squid.vcl \
-s malloc,2048MB \
-w 32,1024,300

>

Varnish acl = for the Purge tab you have to create acl on varnish and that is not part of vcl_recive or fetch , error or etc, so that text box should be added to the custom config tab.

ok, could be done. I'will do it in 0.9 RC

Varnish log Tab = Varnish log is really powerful for debugging and solve problem or bugs, will be really useful to have that on a tab like the varnishtop tab that you can refresh every n seconds and with filter options for something like $ varnishlog -c -o ReqStart 192.0.2.123 and maybe have a drop box with all the -o options supported by varnish.

If any if this options will be added then pfSense will have be careful to add firewall rules to permit the varnishadm port and maybe add the ability to change that port of varnish adm.

This is what it comes to my mind for now, I'm pretty sure that I will come up with something else that it could be useful for everyone that uses varnish on pfSense 2.0.

Please test version 0.8.9. I'm planning to put package varnish in a stable and 'full featured' state.

Actions #3

Updated by Chris Buechler over 9 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF