From c4b6f870ad25d33d6e786ee472c2332cf9c5531f Mon Sep 17 00:00:00 2001 From: randomdan Date: Tue, 8 Mar 2011 18:26:26 +0000 Subject: A good start on docs, WTD --- project2/docs/docbook.xml | 830 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 673 insertions(+), 157 deletions(-) diff --git a/project2/docs/docbook.xml b/project2/docs/docbook.xml index e8b7028..868088d 100644 --- a/project2/docs/docbook.xml +++ b/project2/docs/docbook.xml @@ -1,106 +1,74 @@ - + Project2 - Platform Guide Dan Goodliffe - -
- dan@randomdan.homeip.net -
- - foo + dan@randomdan.homeip.net + Original author 2011 Dan Goodliffe - The complete guide to everything Project2 + The complete guide to everything Project2. - - Project2 - - - Introduction - - - Requirements - - - Compilation - - - Installation - - - Deployment - + - Runtime configuration - - - p2config - 5 - - - p2config - .p2config - Configure runtime behaviour of Project2 application engines. - - - - sysloglevel=<-1...7> - - The level of logging written via syslog3. - - - - consoleloglevel=<-1...7> - - The level of logging written to stderr. - - - - dumpdatadoc=/path/to/file.xml - - Write a copy of the data document produced by p2cgi before sending it to the web server. - This option is available only when compiled with debugging support. - - - - errorcontenttype - - The HTTP Content-Type header producted by p2cgi for an unhandled error document. - - - - errortransformstyle - - The href of the xml-stylesheet p2cgi will write into the processing instruction for an unhandled error document. - - - - + Preparation
- Platform + Introduction + todo +
+
+ Requirements + todo
- - - Execution stages
- Check + Compilation + todo
- Request + Installation + todo
- Present + Deployment + todo
+ + Overview and deployment + + + Project2 + 7 + + + project2 + The Project2 application engine. + + + Script life cycle + + Check + todo + + + Request + todo + + + Present + todo + + + + Engines @@ -110,7 +78,7 @@ p2console - Run one or more Project2 script from the console. + Execute one or more Project2 scripts from the console. @@ -118,6 +86,15 @@ file + + Description + Execute one or more Project2 scripts from the console or from some other process launcher, such as cron. + All stages of execution are supported. + + + Notes + Data source transactions are considered on a script by script basis; a transaction is open when required and always committed or rolled back at the end of a script, before another script is started. + @@ -128,6 +105,13 @@ p2cgi Generate webpages and/or run Project2 scripts from a CGI capable web server. + + Description + The ultimate aim of a p2cgi instance is to generate a document that can be transformed into a webpage. Processing takes part in one of two ways depending on how the instance was requested by the client. + A GET request is the simplest form of request. The request URL is mapped onto a single Project2 script file, and that file is executed to produce a document that is sent to the host web server for transformation. This method supports checks and views; all checks must be passed before any views are executed. A check failure will result in a different (named) Project2 script being executed. No transactions are ever opened on any data source. + A POST request however, provides support for checks and tasks. Again the request URL is mapped to a script and that script is executed. All checks must be passed and then all tasks will be executed. Transactions will be opened as required on data sources and will be committed automatically on successful completion of the script, rolled back otherwise, after which a (named) Project2 script will be presented (see GET request). + Tasks cannot be executed from a GET request and views cannot be (directly) presented from a POST request. The use of transactional data sources is highly recommended, Project2 will tightly integrate its behaviour with them. + @@ -138,9 +122,80 @@ p2fcgi FastCGI implementation of p2cgi. - For all intents and purposes, p2fcgi is a drop in replacement for p2cgi and should be used whenever possible. See your web server configuration documentation for details on running fast CGI applications. - Unless otherwise specified, p2cgi refers to both p2cgi and/or p2fcgi. - p2fcgi has numerous significant benefits over p2cgi. Most notably a near zero startup time, presistent connections to RDBMS datasources and high performance in memory caching of row sets. + + Description + For all intents and purposes, p2fcgi is a drop in replacement for p2cgi, with exactly the same application engine internals, and should be used whenever possible. See your web server configuration documentation for details on running fast CGI applications. + Unless otherwise specified, p2cgi refers to both p2cgi and/or p2fcgi. + p2fcgi has numerous significant benefits over p2cgi. Most notably a near zero startup time, presistent connections to RDBMS datasources and high performance in memory caching of row sets. + + + + + Runtime configuration + + + p2config + 5 + + + p2config + .p2config + Configure runtime behaviour of Project2 application engines. + + + + + sysloglevel=<-1...7> + + The level of logging written via syslog3. + + + + consoleloglevel=<-1...7> + + The level of logging written to stderr. + + + + dumpdatadoc=/path/to/file.xml + + Write a copy of the data document produced by p2cgi before sending it to the web server. + This option is available only when compiled with debugging support. + The dumped document may contain sensitive information (such as passwords); it is your responsibility to delete the file when you're done with it! + + + + errorcontenttype=Some/MimeType; charset=foo + + The HTTP Content-Type header producted by p2cgi for an unhandled error document. + + + + errortransformstyle=transform.xslt + + The href of the xml-stylesheet p2cgi will write into the processing instruction for an unhandled error document. + + + + + + Descripton + .p2config should placed in the web root of a web page and/or the current working directory of a script run from the console. + + + + + config.xml + 5 + + + config.xml + Configure platform specific runtime variables. + + + Descripton + config.xml should placed in the web root of a web page and/or the current working directory of a script run from the console. + @@ -149,11 +204,11 @@ Common - project2:iterate + iterate 7 - project2:iterate + project2-iterate Execute tasks for every record in a row source. @@ -163,15 +218,56 @@ source="row source" filter="row source filter name" + + project2:iterate/parameters/param + name="parameter name" + value="parameter value" + + + project2:iterate/project2:iterate (as above) + + + project2:iterate/project2:if + + + project2:iterate/project2:task + + + Description + Take the records in the named row source and for each one, execute the child tasks specified. + The tasks that can be executed are outside the scope of this document [section] and therefore you should see the documention for the task[s] you wish to execute for more information. + For details on the use of parameters and filters, see project2-view7 as an iterate has exactly the same behaviour in this regard. + An iterate passes nothing to the presentation layer. + Iterations specified as children of an iteration are executed once per parent row. + + + Bugs + The error handling features are completed undocumented! + + + See Also + + project2-if7 + project2-view7 + + + project2-sessionset7 + project2-sessionclear7 + project2-sqlmergetask7 + project2-sqltask7 + project2-sendmail7 + ... + + - project2:view + view 7 - project2:view + project2-view Present data from a row source. @@ -183,15 +279,44 @@ recordname="record xml node name" filter="row source filter name" + + project2:view/columns/column = column name + name="column name" + + + project2:view/parameters/param + name="parameter name" + value="parameter value" + + + project2:view/project2:view (as above) + + + project2:view/project2:if + + + Description + Take rows for the named row source (sqlrows, filerows, etc) and pass their data onto the presentation layer, marked up according to rootname and recordname. filter allows you to specify an optional filter name to the underlying row source, see the row source documentation for details on this. params allow you to pass parameters to the underlying row source. + columns allow you only pass on a subset of columns from the row source, or specify extra columns that would not normally be included. You can also use this to effectively rename columns before passing them on. + The inclusion of more view elements with a view allow use of values in the current row set to be used in generation of child row sets. Note that this will execute the child row source once for every row returned by the current row source and therefore might not be especially efficient. When used with row sources that all the needed flexibility, it might be worth using it to avoid this situation (e.g. consider using filter in conjunction with joins when using project2:sqlrows. The use of columns does NOT effective the availability of values used by child views. + + + Bugs + I can't remember which way round to specify columns. One specifies the name of the row source column name, one specifies the name that is passed to the presentation layer. Nor can I remember how you access row source attributes such as rownum. + + + See Also + project2-if7 + - project2:if + if 7 - project2:if + project2-if Conditionally execute and/or present data. @@ -200,95 +325,486 @@ name="if name" mode="and | or" + + project2:if//if + mode="and | or" + + + project2:if//if/check + mode="and | or" + + + project2:if//view | task | iterate + + + Description + if evaluates its child check and if elements until either a true or false result is determined. Execution takes place in the order specified in the script and halts when the results of further check execution is unimportant to the result (short cutting.) + If, and only if, the checks are evaulated are being true then child view and/or iterate and tasks are executed. + + + See Also + + project2-sqlcheck7 + project2-regexcheck7 + + + project2-if7 + project2-iterate7 + project2-view7 + + + project2-sessionset7 + project2-sessionclear7 + project2-sqlmergetask7 + project2-sqltask7 + project2-sendmail7 + ... + + + + Bugs + The error handling features are completed undocumented! +
Data Sources -
- rdbmsdatasource -
+ + + rdbmsdatasource + 7 + + + project2-rdbmsdatasource + Define a datasource for connecting to an RDBMS database. + + + + project2:rdbmsdatasource + name="datasource name" + + + project2:rdbmsdatasource/masterdsn/connector = connector specific connection string + + + project2:rdbmsdatasource/readonly + preferlocal="true | false" + + + project2:rdbmsdatasource/readonly/dsn/connector = connector specific connection string + host="source hostname" + + + + Description + Defines a named data source for use with a transaction RDBMS database. Readonly actions will be performed against one of the readonly DSNs until such a time that changes are made to the content. The changes will be made on the master DSN, which will be then used for the duration of the current application engine run. Note that an error leading to a rollback will cause the application engine to revert to using a readonly DSN until another change is made. + A readonly datasource is chosen according to the preferlocal and host value of the readonly/dsns. When preferlocal is true, host is matched against the hostname of the local machine (see uname2). If not matching reaodnly/dsn/host is found, masterdsn is used. This mode suits the situation where you might have several web servers, each with a local instance of the database. When perferlocal is false, however, readonly connections are attempted in alphabetical readonly by the host. This would suit have a primary and secondary/failover database. + ANY connection established from any rdbmsdatasource may be used and reused to fulfill another connection. This allows for minimising the overheads in establishing connections. Matching is performed according to connector type and connection string. For this reason, it is advisable to not specify a readonly attribute in a connection string that might match that of master. Doing so would cause two connections to be established when one would otherwise suffice. + Note that the value of dsn/host has NO effect on how a connection is established or where to, only in choosing which connection to use. + + Connectors -
+ ODBC -
-
+ + odbc = odbc connection string + + Connect using unixODBC driver. + + PostgreSQL -
-
-
+ + postgresql = libpq connection string + + Connect using the native PostgreSQL driver. + + + + Bugs + Despite the amount of textual duplication required in a simple configuration with failover, the actual amount of control you get is still quite poor. For example, there is no way to specify any form of round robin DB pool or use any sort of load balancing of a DB cluster. As such, expect changes to this component. + + +
+
+ Helpers + + + curlhelper + 7 + + + project2-curlhelper + Provide support for retrieving data with libcurl3. + + + + project2:curlhelper + url="resource url" + useragent="user agent string" + cookiejar="cookie jar file path" + proxy="proxy address" + method="request method" + username="request username" + password="request password" + timeout="timeout (ms)" + + + + Notes + All values are passed verbatim (after suitable type coercion) to libcurl3, see curl_easy_setopt3 for details. Note also, that the mapping between Project2 variables and libcurl settings may not be one to one and that some settings are only useful with some protocols. + An instance of curlhelper can't exist on its own, only as part of something else. + + + Bugs + Not all libcurl settings are supported. + + + + + streamrows + 7 + + + project2-streamrows + Provide common functionality for processing a byte stream into record and field data. + + + + project2:streamrows + + + + Notes + An instance of streamrows can't exist on its own, only as part of something else. + + + Bugs + Many features are missing from stream rows, which still make it difficult or impossible to support some common file formats, procrows can sometimes fill this gap, but it starts getting messy. + +
Row sources -
- xslrows -
-
- xmlrows -
-
- sqlrows -
-
- filerows -
-
- fsrows -
-
- urlrows -
-
- procrows -
+ + + xslrows + 7 + + + project2-xslrows + Create a row set from an XML or HTML document and a set of XPaths. + + + + project2:xslrows + name="row set name" + url="document URL" + html="true | false" + warnings="true | false" + + + project2:xslrows/namespace + prefix="namespace prefix" + url="namespace url" + + + project2:xslrows/filterview + name="filter name" + root="xpath of record root" + + + project2:xslrows/filterview/field + name="field name" + xpath="xpath of field value" + + + + Description + Creates a row set from an XML or HTML document based on evaluations of XPaths against that tree. + Each filterview defines an extractable row set. The root specifies what constitutes a record such as /html//h1 would create a record in the row set for all level 1 headings in a HTML document. You could then create fields in that record based on XPaths such as @class and .//text() to extract the heading's CSS style and wording. + Any number of named filterviews can be created, with any number of named fields with them. A user of this row set must specify a filter. + + + Notes + xslrows use project2-curlhelper7 and thus support all its attributes for accessing the document resource. + Large documents may consume a lot of memory and may be held in memory until the end of the application engine iteration. See project2-xmlrows7 for a less fully featured alternative that does not have this issue. + + + Example + + Extract headings and heading styles from an HTML document + +<project2:xslrows name="example" url="file:///tmp/doc.html" html="true"> + <filterview name="heading1s" root="/html//h1"> + <field name="text" xpath=".//text()"> + <field name="style" xpath="@class"> + </filterview> +</project2:xslrows> + + + + + Bugs + This component is badly named and should be renamed to xpathrows. + + + See Also + project2-curlhelper7 + + + + + xmlrows + 7 + + + project2-xmlrows + Create a row set while reading through an XML resource. + + + + + sqlrows + 7 + + + project2-sqlrows + Create a row set from an SQL select. + + + + project2:sqlrows + name="row set name" + datasource="the name of an rdbmsdatasource" + + + project2:sqlrows/sql = SQL text | <filter> | <param> + + + project2:sqlrows/sql/filter = SQL text | <param> + name="filter name" + + + project2:sqlrows/sql//param + name="parameter name" + + + + Description + This component constructs an SQL statement for execution against an RDBMS data source. + SQL text within a filter element is condition upon that filter being specified by a row set user. Many filter elements can have the same name and upon its use, all elements will be included in the result SQL. Parameters specified with the param may also shared the same name, in which case the same value will be bound to all positions in the query. + Any part of the query can be part of the filter element and can therefore include extra columns (for example, returning a match accuracy only when a free text search filter is used) or an order by clause which might be different depending on which filter is used. You can also join extra tables or sub-selects if you need to. + param elements will be automatically translated into SQL text suitable for using bound parameters with your chosen RBDMS data source driver. No other changes to the SQL are made. For the purpose, it is assume that ' (single quote) is used to mark the beginning and end of a string literal, and '' (two single quotes) are used to escape a single quote within a string literal. + + + Example + + Select all or some employees from an SQL database + +<project2:sqlrows name="example" datasource="staffdb"> + <sql> + SELECT empno, firstname, lastname, deptno + FROM employees + <filter name="bydept"> + WHERE deptno = <param name="deptno" /> + </filter> + </sql> +</project2:sqlrows> + + + When no filter is specified by a row set user, returns all employees in the employees table. When the bydept filter is specified along with a department number parameter, only employees in that department are returned. + This example assumes the existance of the employees table with suitable columns. + + + Bugs + There is no concept of a default filter, which might be handy, although there is nothing to stop you defining a filter named default and specifying that. + Only one filter can be used at any time. So, in our example, if you wanted to filter by deptno, or lastname or (deptno and lastname) you would have to define three different filters. While this doesn't place any limits on what you can do, it might lead you to duplicating SQL which would otherwise not be required. + + + See Also + project2-rdbmsdatasource7 + + + + + filerows + 7 + + + project2-filerows + Create a row set from a character separated local file. + + + + + fsrows + 7 + + + project2-fsrows + Create a row set from files and directories on the local filesystem. + + + + + urlrows + 7 + + + project2-urlrows + Create a row set from a character separated remote resource. + + + + + procrows + 7 + + + project2-procrows + Create a row set from the character separated output of another program. + +
Tasks -
- sessionset -
-
- sessionclear -
-
- sqlmergetask -
-
- sqltask -
-
- sendmail -
+ + + sessionset + 7 + + + project2-sessionset + Set a value in the current session. + + + + + sessionclear + 7 + + + project2-sessionclear + Remove a value from the current session. + + + + + sqlmergetask + 7 + + + project2-sqlmergetask + Merge row set data into an RDBMS table. + + + + + sqltask + 7 + + + project2-sqltask + Execute an arbitory DML statement. + + + + + sendmail + 7 + + + project2-sendmail + Send an email. + +
Variables -
- literal -
-
- config -
-
- parent -
-
- uri -
-
- param -
-
- session -
+ + + literal + 7 + + + project2-literal + A literal value. + + + + + config + 7 + + + project2-config + Retrieve a value from the platform configuration. + + + + + parent + 7 + + + project2-parent + Retrieve a value from a current in execution row set field or attribute. + + + + + uri + 7 + + + project2-uri + Retrieve a value from the URL of the original request. + + + + + param + 7 + + + project2-param + Retrieve a value from the parameters of the original request. + + + + + session + 7 + + + project2-session + Retrieve a value from the current session. + +
Checks -
- regexcheck -
-
- sqlcheck -
+ + + regexcheck + 7 + + + project2-regexcheck + Check a value against a regular expression. + + + + + sqlcheck + 7 + + + project2-sqlcheck + Check a value against an RDBMS data source. + +
-- cgit v1.2.3