| home | updates | download | manual | documents | feedback | search | ITS more |
|
HTTP proxy/server
replace-[i][qr]head:name:body
replace headers of "name:" with "name:body"
in forwarded request/response message.
This is equivalent to the combination of
kill-head and add-head.
kill-tag: listOfTags
disable a tag listed in listOfTags when it it used in
a text/html response from a server.
ver:1.0
act as a HTTP/1.0 client/server
svver:1.0
act as a HTTP/1.0 client against servers (send request in HTTP/1.0)
clver:1.0
act as a HTTP/1.0 server against clients
(do not use chunked encoding in response)
acc-encoding:encoding [-thrugzip]
Accept-Encoding header to be sent to server.
This is applied to DeleGate which does some interpretation of content
with MOUNT, CHARCODE, CACHE, etc. To do such interpretation, the content
(response body) is not to be encoded in a format unknown to DeleGate.
"identity" specifies disabling any encoding of content in the server.
"-thrugzip" specifies forwarding Accept-Encoding:gzip from client to server.
If the program "gzip" is not available on the host of DeleGate
(i.e. not found in LIBPATH),
"identity" is sent regardlessly.
gen-encoding:encoding [gzip]
The encoding applied to the content sent from DeleGate to client.
Only "gzip" is available in the current implementations.
"identity" and others disable any encoding.
tout-wait-reqbody:period [30]
max. period to wait the first data in request body.
tout-in-reqbody:period [15]
max. period to wait next data in request body.
(adjustable for a slow filter program at the client side)
tout-buff-reqbody:period [5]
max. period to do buffering request message to server
tout-buff-resbody:period [8]
max. period to do buffering response message to client
tout-cka:period [5]
max. period to keep a connection with the client alive.
(the timeout is ten times longer for the first connection from each client host)
max-cka:number [50]
max. number of requests to be relayed on a single connection
in keep-alive.
(ten times larger value is given to the first connection from each client host)
This is applied to the communication over a connection of SSLtunnel by the CONNECT method too. A pair of upstream and downstream data is regarded as a pair of request and response on HTTP, then the maximum number of such pairs is restricted.
max-ckapch:number [8]
max. number of connections in keep-alive at a time for each client host.
max-reqline:length [8k]
max. length of request line to be accepted.
max-reqhead:length [12k]
max. length of request header to be accepted.
max-gw-reqline:length [512]
max. length of request line to be forwarded to another server
in another protocol.
max-hops:number [20]
max. number of hops in the chain of HTTP proxies.
tout-resp:period
[TIMEOUT=io]
max. period to wait response from server.
tout-pack-intvl [10.0]
max. interval between packets relayed on SSLtunnel
by the CONNECT method.
halfdup
Forbid full-duplex usage of SSLtunnel by the CONNECT method.
urlesc[:escChars] [empty]
the set of characters in request URL to be escaped by "%XX"
notation before any processing.
As a special case, HTTPCONF="urlesc" means HTTPCONF="urlesc:<>".
mypro.xyz:{forw,thru,stop}
By default, an access to the virtual domain name
"http://mypro.xyz"
is interpreted as an access to your proxy server.
HTTPCONF="mypro.xyz:forw" enables forwarding the request to
upstream proxies by prefixing the hop count.
HTTPCONF="mypro.xyz:thru" disables this interpretation and
forward the request to the upstream proxy if exists.
HTTPCONF="mypro.xyz:stop" stops this interpretation and forwarding at all.
Another way for restriction of access to mypro.xyz is using AUTHORIZER
as AUTHORIZER="-list{User:Pass}:*:*.mypro.xyz:*".
proxycontrol[:{on|off}]
consume substring following "?_?" in request URL as control information for
the DeleGate; when DeleGate get request URL?_?proxycontrol,
only URL part is forwarded to the server
and proxycontrol part is (possibly) used by DeleGate.
cka-cfi
make connection with the client keep-alive even with external filter
(FCL, FTOCL).
nolog:listOfCodeType[:connMap]
listOfCodeType == [ respCode / ][ Type [ / subType ] ]
specify response code or Content-Type of response message not to be logged in access log (PROTOLOG).
xferlog:ftp
record FTP/HTTP transactions in xferlog format too.
bugs:listOfBugs
listOfBugs is a list of features to be disabled
to bypass possible bugs as follows:
bugs:thru-304
disable the conversion from "304 Not Modified" to "200 Ok" in the message
as the response to a conditional request with the "If-Modified-Since" header.
DeleGate with external filters tries to return a HTTP response messages with
a body (with the code "200 Ok") when it is filtered (and possibly rewritten)
by the filters even if the body should be returned as empty (304 Not Modified)
based on the modification date (Last-Modified) of the target data.
This is necessary to return data rewritten dynamically by filters, but it
disables the merit by the conditional request and the "304" response.
"thru-304" turns the above conversion off and let DeleGate pass through
"If-Modified-Since" request from clients and the "Last-Modified" and
"304 Not Modified" response from servers.
svauth:no-basic
Stop forwarding Basic Authorization (which contains cleartext password)
from client to server.
If the server supports Digest authentication, then the DeleGate do it
by proxy of the client.
svauth:less-basic
Postpone forwarding Basic Authorization from client to server until
the server requires the Basic Authentication.
session:cookie
Enable session management based on Cookie.
The session identifier
is passed to CFI/CGI programs in
environment variable "X_COOKIE_SESSION", and
can be logged into PROTOLOG.
Example:
HTTPCONF=search:/path/of/searchScript
MOUNT="/bin/* cgi:/path/of/cgi-bin/* search:-"
MOUNT="/* file:/path/of/data/*"
HTTP proxy/server
-
Example: DeleGate as an HTTP proxy
-
firewall% delegated -P8080 SERVER=http
Then use this DeleGate from your client on the internal host, specifying "firewall:8080" as the proxy for HTTP, HTTPS (Security), FTP, Gopher, Wais, and so on.
Example: cascaded DeleGate as an HTTP proxy
-
firewall% delegated -P8888 SERVER=delegate RELIABLE=internal
internal% delegated -P8080 SERVER=http MASTER=firewall:8888
Run a generalist DeleGate on the firewall which only accepts request from internal host, then run a specialist on internal which use the generalist on firewall host. A generalist can be shared as an upstream DeleGate from multiple DeleGates of arbitrary protocol.
Example: DeleGate as an origin HTTP server
A file with a name with ".cgi" extension is treated as a CGI script.
Also you can use arbitrary name of CGI scripts under a specified
directory with a MOUNT like:
MOUNT="/xxx/* cgi:/path/of/cgi-bin/*"
Example: DeleGate as a CGI program
-
You can use DeleGate as a CGI program from a HTTP server. For example,
specify in the "httpd.conf" file of your HTTP server(A) as follows.
-
Exec /other/* /path/of/cgi-delegate
-
#!/bin/sh
HTTP Transfer Log Format
-
The format of PROTOLOG for HTTP protocol can be modified with a
optional format specification postfixed after the LogFilename as
PROTOLOG="LogFilename:format".
In this case, default file name can be omitted.
For example, PROTOLOG=":%X" specifies making a NCSA like
extended common logfile format.
The default format is
PROTOLOG=":%X %D" in version 10 while it was
PROTOLOG=":%C %D" in version 9.
| %C | -- common logfile format of CERN-HTTPD |
| %c | -- same as %C except the date is recorded in millisecond precision |
| %D | -- extension by DeleGate (connTime+relayTime:status) |
| %X | == '%C "%r" "%u"' common logfile format with Referer and User-Agent |
| %r | -- Referer field in the request message |
| %u | -- User-Agent field in the request message |
| %S | -- status of CONNECTion to the server (c,m,p,d,s,v) |
| %s | -- session Id by HTTPCONF=session |
| %As | -- session Id in Digest Authorization |
%{field}| -- arbitrary field in the request message
| |
The session identifier put by "%s" is generated by specifying HTTPCONF=session:cookie option while "%As" is generated by AUTHORIZER=-dgauth option. The format of session identifier is as this:
-
time.pid.proc#+conn#[+req#].reqnum
For example, "031114-173045.1234.5+6+7.9" means that it is the 9th request from the client in the session started at 17:30:45 on November 14 by the process of PID=1234. The start of the session is recorded in LOGFILE as this:
-
11/14 17:30:45 [1234] 5+6/7: NewSession 031114-173045.1234.5+6+7.0
Note that the reqnum part may not be unique because of parallel or pipelined requests generated by a client as the owner of the session. Note that the reqnum part for "%As" may not be incremented on each request because the container of session identifier, the opaque" parameter in Digest Authentication in this case, may not be updated on each request.
HTTPCONF parameter == HTTPCONF=what:conf
- welcome:listOfWelcomeFiles
-
-- default: welcome.{dgp,shtml,html,cgi},index.{dgp,shtml,html,cgi},-dir.html
A list of index files or CGI scripts which should be used for URLs of directories ending with "/" like "/path/". This is a list of candidate file names. The list may be ended with "-dir.html" which means a built-in index generator. If the list is empty, empty data is substituted for index data. - search:pathOfSearchScript
-
-- default: none
The path of a CGI script to be applied for URLs with search part like "/path?search". This is a global specification applied for all URLs. Also you can specify a local search script for each MOUNT point like-
MOUNT="/path2/* /root/of/path2/* search:script2".
- nvserv:noauto | auto | alias | gen | none
-
-- default: HTTPCONF=nvserv:noauto
Automatically detects virtual servers in MOUNT parameters and notate each of them as a MOUNT rule to be applied only to a virtual server. It can be done manually by specifying "nvserv" MountOption for each MOUNT parameter. "nvserv:alias" means detecting target servers of host names with common IP-addresses and notate them as virtual servers. "nvserv:gen" means notating MOUNT parameters with "genvhost" MountOption as virtual servers. "nvserv:auto" is equivalent to "nvserv:alias,gen". The guessing can be overridden by explicitly specifying the avserv MountOption for each MOUNT parameter. "nvserv:none" disables any treatment of virtual servers which are detected automatically or specified explicitly by the "nvserv" MountOption. - methods:listOfAcceptableMethods
-
-- default: methods:OPTIONS,GET,HEAD,POST,PUT,...
-- See the output to LOGFILE with HTTPCONF=methods:"+"
Limit or add HTTP methods to be accepted.
Example:
-
HTTPCONF=methods:GET,HEAD -- accept only GET and HEAD
HTTPCONF=methods:-POST,-PUT -- don't accept POST and PUT
HTTPCONF=methods:+,NEWMETHOD1 -- add NEWMETHOD1 to be accepted
HTTPCONF=methods:* -- accept any methods
- rvers:listOfAcceptableResponseVersions
-
-- default: rvers:HTTP
Add versions in response message from HTTP servers.Example:
- post-ccx-type:listOfTypes
-
-- default: post-ccx-type:x-www-form-urlencoded
Specify the list of names of Content-Type of request message to which conversion by CHARCODE is applied.Example:
-
HTTPCONF=post-ccx-type:+,multipart/form-data
- cryptCookie:listOfCookies:cryptKey
-
listOfCookies == attributes[@domains]
attributes == attribute | {attribute,attribute,...}
domains == domain | {domain,domain,...}
domain == [.]domainName
cryptKey == string | %a | %Pencrypt specified attributes in a Set-Cookie response to be stored in a client, then decrypt and forward the Cookie request only to the originator of the Cookie. An attribute in a Cookie is specified as "attribute@host" or "attribute@.domain". In the former case, a cookie generated by a host is encrypted and echoed to host only. In the latter case, a cookie generated by hosts in the domain can be echoed to hosts in the domain. The special string "%a" in cryptKey is substituted by the IP-address of the client. This makes the crypted Cookie be usable only by clients on the host of the IP-address.
Example:
- kill-[i][qr]head: listOfHeaders
- erase header fields listed in listOfHeaders before forwarding
a request/response message to server/client. "kill-qhead" is applied
only to request message to server and "kill-rhead" is applied only to
response message to client.
If "i" is prefixed as "kill-iqhead:Pragma,Cache-Control", the specified
fields are erased before DeleGate interpret it as the HTTP headers.
Example:
- add-[i][qr]head:name:body
- add a header name:body to forwarded request/response message.
Client's identity information can be inserted into body string
with "%format" notation.
If "i" is prefixed as "add-ihead:Pragma:no-cache", then the specified field
with the body is added to the input to be interpreted by DeleGate
as a HTTP header from the client or the server.
Example:
-
HTTPCONF="add-qhead:X-Forward-For:%a"
Example:
Example:
-
HTTPCONF=kill-tag:SCRIPT,APPLET
This is applied to the communication over a connection of SSLtunnel by the CONNECT method too. A pair of upstream and downstream data is regarded as a pair of request and response on HTTP, then the maximum number of such pairs is restricted.
specify response code or Content-Type of response message not to be logged in access log (PROTOLOG).
Example:
-
HTTPCONF="nolog:302,304,image"
- no-gzip ... disable Content-Encoding:gzip
- no-keepalive ... disable Connection:Keep-Alive
- no-keepaliveproxy ... disable Connection:Keep-Alive with client side proxy
- no-chunked ... disable Transfer-Encoding:chunked
- no-flush-chunk ... disable flushing response after each chunk
- kill-contleng ... erase original Content-Length in chunked encoding
- add-contleng ... add or update Content-Length even in chunked encoding
- do-authconv ... enable Authentication conversion from client's Basic to server's Digest
- no-keepalive ... disable Connection:Keep-Alive
Example:
MOUNT="/bin/* cgi:/path/of/cgi-bin/* search:-"
MOUNT="/* file:/path/of/data/*"
FILETYPE parameter == FILETYPE=suffix:gopherType:altText:iconName:contentType -- default: FILETYPE=".txt:0:TXT:text:text/plain" FILETYPE=...
-
Define file name to content-type mapping.
- gopherType:
- 0 - text, 1 - directory, 4 - BinHex, 6 - uuencode, 9 - binary, g - gif, I - image, ...
- altText:
- alternative text which should be displayed for the icon image in text only displays (embedded into IMG tag in HTML text like <IMG ALT="altText" ... >)
- iconName:
- one of binary, binhex, compressed, directory, document, ftp, gzip, image, index, index2, movie, sound, tar, telnet, text, unknown, uu (see http://delegate/-/builtin/icons/)
- contentType:
- text/plain, image/gif, ... (see RFC2045)
Example:
CGIENV parameter == CGIENV=name[,name]* -- default: CGIENV="*"
-
A list of environment variables to be passed to CGI program.
MountOptions for HTTP-DeleGate
-
CONDITIONS:
- vhost={HostList} -- virtual host matching and rewriting.
- true if the value of the "Host:" field in the request message is included in the HostList for request rewriting, and true unconditionally for response rewriting. It supports virtual hosting by a special rewriting for response, that is, any full-URL of a real host in a response message will be rewritten to that of a virtual host.
- avhost={HostList} -- address based virtual hosting
- equivalent to the "vhost" option.
- nvhost={HostList} -- name based virtual hosting
-
similarly to "vhost" option, "nvhost" is used to configure virtual hosting
except that this option is restricted to create name based virtual hosts
which are distinguished each other only by their host names.
Each virtual host is identified textually by its name (thus no "-" prefix
for each host is necessary to force textual-only matching which is necessary
with the "vhost" option).
For example, the following two MOUNT parameters are equivalent to each other.
MOUNT="/* http://server/* vhost=-www.domain"
MOUNT="/* http://server/* nvhost=www.domain"
A special option "nvhost=-thru" can be used to ease forwarding a virtual domain name indicated by a client to a server. The option means matching with virtual domain name to be sent to the target server. By default, the domain name can be specified explicitly with the nvserv or the target server name in the rURL of this MOUNT rule by default. For example, the following three MOUNT parameters are equivalent mutually.
MOUNT="/* http://domain/* nvhost=-thru,rserv=server"
MOUNT="/* http://server/* nvhost=-thru,nvserv=domain"
MOUNT="/* http://server/* nvhost=domain,nvserv=-thru"
- qmatch=pattern -- pattern matching in the request header
- true if the pattern matches with string in the request header. (ex. qmatch=*User-Agent:*compatible;%20MS*)
- dstproto={ProtoList} -- to the server of specified protocol
- true if the protocol of the destination server is in the ProtoList.
- method={methodList} -- request method
- true if the access method of the current request is included in the methodList.
- asproxy
- true if the DeleGate is called as a proxy server that is the requested URL is in full URL format.
- !asproxy
- true if the DeleGate is not accessed as a proxy server in the request.
- withquery
- true if the requested URL has "?query" component
- authorizer=authServList
- the AUTHORIZER which is to be applied to this MOUNT point. If AUTHORIZER is specified in both MountOption and command line option, the one in MountOption is used.
- moved[={300|301|302|303}]
- don't relay to rURL but return response for redirection as "302 Moved" with "Location:rURL". Redirection within the same server can be eased using the abbreviation of host-name in the rURL like MOUNT="/path1 ///path2 moved".
- referer
- Apply the rewriting only to the Referer: field to be forwarded. This is an synonym of "where=ref".
- useproxy[=proxyURI]
- generate "305 Use Proxy" response message. If proxyURI is omitted or given as "direct", the response message is sent with Set-Proxy field as "Set-Proxy: DIRECT". Otherwise the URI is set in the field as "Set-Proxy: SET; proxyURI=proxyURI".
- realm=realmString
- specify the realm of authentication.
- forbidden
- reject the request as forbidden (synonym of "rcode=403")
- unknown
- reject the request as unknown (synonym of "rcode=404")
- rcode={300|301|302|303|304|305|306|403|404}
- return the response with the specified status code, which can be useful for customizing error messages.
- onerror[=listOfCodes]
- this MOUNT entry is used to substitute a response message when an error occurred. It is applied If the response status code for the request shows an error (4xx or 5xx), and the status code is in the listOfCodes if it is specified. For example with MOUNT="/path/* file:/tmp/autherr.cgi onerror={401,407}", the output from /tmp/autherr.cgi is sent when authentication error occurred in access to URLs under "/path/".
- robots={no|ok}
- allow or disallow retrievals from robots. The default value for NNTP and FTP is "no", while it is "ok" for other protocols.
- nvserv | nvserv=host -- forwarding to a name based virtual server
- specify MOUNTing the target server as a name based virtual hosting server. The virtual host name to be sent to the server can be specified as "nvserv=host" or it is the host name part in the rURL by default. With this option, the target server is MOUNTed only as a virtual server of the host name while other servers, of alias names of the host or IP-addresses of the host, are not MOUNTed. This means that the MOUNT rule will be applied for rewriting of URL in a HTTP response message only if the hostname of a URL matches textually with the virtual host name of the target server. The virtual hostname from the client can be passed through to the server with "nvserv=-thru".
- avserv | avserv=host -- forwarding to an IP-address based virtual server
- notate that the target server is not a name based virtual hosting server. With this option, all of servers of alias names or IP-addresses of the target server are MOUNTed by this MOUNT rule. It has been the default behavior of MOUNT but it can be changed with the option HTTPCONF="nvserv:auto" to automatically detect virtual servers in a set of MOUNT parameters. This "avserv" option could become necessary to notate a non-virtual server which is automatically guessed as a virtual server.
- rserv=host[:port] -- real target server
- specify the real host name or IP-address and port number of the target
server toward which the HTTP connection is established.
This option is used to do MOUNT a virtual hosting server of which virtual
host name is specified in the rURL rather than in the
"nvserv=host" option.
For example, the following two MOUNT rules are equivalent to each other.
MOUNT="/v/* http://www.domain/* rserv=192.168.1.123"
MOUNT="/v/* http://192.168.1.123/* nvserv=www.domain"
Specifying the "rserv" option implies that the target server is a virtual server as if notated with the "nvserv" option. - genvhost=host (obsoleted by nvserv and rserv options)
- Specify the hostname which will be received by the target server as its virtual hostname. It is set in the "Host: host" field in the request message to be forwarded to the server. When the request is to be forwarded via a HTTP proxy, it is set in the request URL as http://host/... The default Host field sent to the server is derived from the rURL part of the MOUNT parameter which specifies the target server. For example for MOUNT="/* http://hosti:8080/*", "Host: hosti:8080" will be forwarded to the server (which is running at the port hosti:8080). To through pass the Host field in the request from a client to the server, specify as "genvhost=-thru".
- ftosv=-cc-charCode
- convert charset in HTTP header forwarded to server into charCode (jis|sjis|euc|utf8)
- pathext=string
- when DeleGate as an origin HTTP server searches a file for requested URL, the path name extended with the specified string is retrieved first. For example with "pathext=-ja", "index-ja.html" is retrieved prior to "index.html". It can be useful in combination with other MOUNT conditions, like MOUNT="/* file:data/www/* nvhost=www.delegate.jp,pathext=-ja"
CONTROLS:
Example: virtual hosting, acting as multiple HTTP servers
-
The target server or local directory is switched by with which name
the DeleGate is referred (in the "Host:vhost" header field).
In this example, requests for "http://dom1.com" and "http://dom2.org"
are forwarded to each corresponding server, while requests for
"http://dom3.net" or any other name are retrieved in each corresponding
local directory.
MOUNT="/* http://wwwA/* nvhost=dom1.com"
MOUNT="/* http://wwwB/* nvhost=dom2.org"
MOUNT="/* file:data/wwwC/* nvhost=dom3.net"
MOUNT="/* file:data/www/*"
Example:
-
"useproxy", "method", "dst" and "withquery" options are introduced
originally to refuse potentially troublesome accesses which may invoke
CGI programs in target servers. For example, to refuse any request
with POST method, or with URL including "?":
MOUNT="http:* = method=POST,asproxy,useproxy"
MOUNT="http:* = withquery,asproxy,useproxy"
AUTH parameters for HTTP-DeleGate
- AUTH=origin:auth
-
-- default: AUTH=origin:ident
Like AUTH=proxy, specify using the value in "Authorization" field in the HTTP header to authenticate the user, except that accessibility to a DeleGate as a "HTTP origin server" is checked in this case.Example:
-
AUTH=origin:auth RELIABLE="*@localhost"
Any user who can login to the localhost of this DeleGate, giving a correct pair of user name and password in Authorization, will be authorized to access. - AUTH=forward[:{by,for,ver,*}]
- Put identification information of the source host in the Forwarded
field in a HTTP request header like:
-
Forwarded: by Me (Version) for Client
- AUTH=viagen[:hostIdentifier]
- Specify the identifier of the host of this DeleGate, which is put into
the Via field in forwarded HTTP message headers as follows:
-
Via: protocol-version hostIdentifier ( comment )
If '%' character is used in a hostIdentifier, it is interpreted as the format for authString below. - AUTH=authgen:basic:authString
- Generate "Authorization: Basic authString"
in a HTTP request header to be forwarded to a server,
if it does not have an original Authorization field from a client.
The authString should be "userName:passWord".
The following special string stand for attributes of clients.
%u -- user name got using Ident protocol %h -- host name of the client got from the socket %i -- host name of the network interface to the client %I -- like %i but use the value of "Host:" if given in HTTP %a -- host address of the client %n -- network address of the client %H -- hostname of the DeleGate %M -- the ADMIN of the DeleGate %A -- generated string by "CMAP=string:authgen:mapSpec" %U -- username part of client's [Proxy-]Authorization: username:password
%P -- password part of client's [Proxy-]Authorization: username:password
Example:
-
When the firewall have two network interfaces and internal
and external hosts access from different interface, then
they can be distinguished by the name of interface.
-
AUTH="authgen:basic:%i:%h"
A generated password is formatted as "passWord/%i" and a DeleGate rejects incoming requests with an Authorization field of such pattern. Thus forged password cannot pass the DeleGate on the host "%i".
- AUTH=pauthgen:basic:authString
- Generate "Proxy-Authorization: Basic authString"
like AUTH=authgen.
Note: obsoleted by MYAUTH="user:pass:http-proxy".Example:
-
Consume Proxy-Authorization in a request message from a client then
forward it to an upstream proxy as is (by authString == %U:%P)
AUTH=proxy:pauth AUTH="pauthgen:basic:%U:%P" PROXY=... - AUTH=fromgen:fromString
- If specified, "From: fromString" will be put in the HTTP
request if the original header does not have an original
From field. If fromString is omitted, the default value
is "%u@%h".
- AUTH=log:remoteHost:identUser:authUser
- Specify contents of the client information part in common
logfile format of HTTP servers. The default value is
AUTH="log:%h:%u:%U".
%F -- E-mail address in From field
%L -- local part of From: local@domain field
%D -- domain part of From: local@domain field
%U -- username part of Authorization: username:password
%P -- password part of Authorization: username:password
%Q -- "for clientFQDN" part of Forwarded: field
Example: