Advanced Zope Scripting

Zope manages your presentation, logic and data with objects. So far, you've seen how Zope can manage presentation with DTML, and data with files and images. This chapter shows you how to add Script objects that allows you to write scripts in Python, and Perl through your web browser.
What is logic and how does it differ from presentation? Logic provides the actions that change objects, send messages, test conditions and respond to events, whereas presentation formats and displays information and reports. Typically you will use DTML to handle presentation, and Zope scripting with Python and Perl to handle logic.

Zope Scripts

Zope Script objects are objects that encapsulate a small chunk of code written in a programming language. Currently, Zope provides Python-based Scripts, which are written in the Python language, and Perl-based Scripts which are written in the Perl language. Script objects are new as of Zope 2.3, and are the preferred way to write programming logic in Zope.
So far in this book you have heavily used DTML Methods and Documents to create simple web applications in Zope. DTML allows you to perform simple scripting operations such as string manipulation. For the most part, however, DTML Methods should be used for presentation. DTML Methods are explained in Chapters 4, "Dynamic Content with DTML", and Chapter 8, "Variables and Advanced DTML".
Here is an overview of Zope's scripts:

Python-based Scripts

You can use Python, a general purpose scripting language, to control Zope objects and perform other tasks. These Scripts give you general purpose programming facilities within Zope.

Perl-based Scripts

You can use Perl, a powerful text processing language, to script Zope objects and access Perl libraries. These scripts offer benefits similar to those of Python-based Scripts, but may be more appealing for folks who know Perl but not Python, or who want to use Perl libraries for which there are no Python equivalents.

You can add these scripts to your Zope application just like any other object.

Calling Scripts

Zope scripts are called from the web or from other scripts or objects. Almost any type of script can be called by any other type of object; you can call a Python-based Script from a DTML Method, or a built-in method from a Perl-based Script. In fact scripts can call scripts which call other scripts, and so on. As you saw in Chapter 4, "Dynamic Content with DTML", you can replace a script with a script implemented in another language transparently. For example if you're using Perl to perform a task, but later decide that it would be better done in Python, you can usually replace the script with a Python-based Script with the same id.
When you call a script, the way that you call it gives the script a context in which to execute. A script's context is important. For example, when you call a script you usually want to single out some object that is central to the script's task. You would call the script in the context of the object on which you want it to carry out its task. It is simpler to just say that you are calling the script on the object.

Calling Scripts From the Web

You can call a script directly from with web by visiting its URL. You can call a single script on different objects by using different URLS. This works because by using different URLs you can give your scripts different contexts, and scripts can operate differently depending on their context. This is a powerful feature that enables you to apply logic to objects like documents or folders without having to embed the actual code within the object.
To call a script on an object from the web, simply visit the URL of the object, followed by the name of the script. This places the script in the context of your object. For example suppose you have a collection of objects and scripts as shown in Figure 8-1.

Figure 8-1 A collection of objects and scripts
To call the feed script on the hippo object you would visit the URL Zoo/LargeAnimals/hippo/feed To call the feed script on the kangarooMouse object you can visit the URL Zoo/SmallAnimals/kangarooMouse/feed. These URLs place the feed script in the context of the hippo and kargarooMouse objects, respectively.
Zope uses a URL as a map to find what object and what script you want to call.
Zope breaks apart the URL and compares it to the object hierarchy, working backwards until it finds a match for each part. This process is called URL traversal. For example, when you give Zope the URL Zoo/LargeAnimals/hippo/feed, it starts at the root folder and looks for an object named Zoo. It then moves to the Zoo folder and looks for an object named LargeAnimals. It moves to the LargeAnimals folder and looks for an object named hippo. It moves to the hippo object and looks for an object named feed. The feed script can't be found in the hippo object and is located in the Zoo folder by a process called acquisition.
Acquisition does two things. First it tries to find the object in the current object's containers. If that doesn't work it backs up along the URL path and tries again. In this example Zope first looks for the feed object in hippo, then it goes to the first container, LargeAnimals, and then to the next container, Zoo, where feed is finally found.
Now Zope has reached the end of the URL. It calls the last object found, feed. The feed script operates on its context which is the second to last object found, the hippo object. This is how the feed script is called on the hippo object.
Likewise you can call the wash method on the hippo with the URL Zoo/LargeAnimals/hippo/wash. In this case Zope acquires the wash method from the LargeAnimals folder.
More complex arrangements are possible. Suppose you want to call the vaccinate script on the hippo object. What URL can you use? If you visit the URL Zoo/LargeAnimals/hippo/vaccinate Zope will not be able to find the vaccinate script since it isn't in any of the hippo object's containers.
The solution is to give the path to the script as part of the URL. This way, when Zope uses acquisition to find the script it will find the right script as it backtracks along the URL. The URL to vaccinate the hippo is Zoo/Vet/LargeAnimals/hippo/vaccinate. Likewise, if you want to call the vaccinate script on the kargarooMouse object you should use the URL Zoo/Vet/SmallAnimals/kargarooMouse/vaccinate.
Let's follow along as Zope traverses the URL Zoo/Vet/LargeAnimals/hippo/vaccinate. Zope starts in the root folder and looks for an object named Zoo. It moves to the Zoo folder and looks for an object named Vet. It moves to the Vet folder and looks for an object named LargeAnimals. The Vet folder doesn't contain an object with that name, but it can acquire the LargeAnimals folder from its container, Zoo folder. So it moves to the LargeAnimals folder and looks for an object named hippo. It then moves to the hippo object and looks for an object named vaccinate. Since the hippo object does not contain a vaccinate object and neither do any of its containers, Zope backtracks along the URL path trying to find a vaccinate object. First it backs up to the LargeAnimals folder where vaccinate still can't be found. Then it backs up to the Vet folder. Here it finds a vaccinate script in the Vet folder. Since Zope has now come to the end of the URL, it calls the vaccinate script in the context of the hippo object.
When Zope looks for a sub-object during URL traversal, it first looks for the sub-object in the current object. If it can't find it in the current object it looks in the current object's containers. If it still can't find the sub-object, it backs up along the URL path and searches again. It continues this process until it either finds the object or raises an error if it can't be found.
This is a very useful mechanism, and it allows you to be quite expressive when you compose URLs. The path that you tell Zope to take on its way to an object will determine how it uses acquisition to look up the object's scripts.

Calling Scripts from other Objects

You can call scripts from other objects. For example, it is common to call scripts from DTML Methods.
As you saw in Chapter 8, "Variables and Advanced DTML", you can call Zope scripts from DTML with the call tag. For example:

        <dtml-call updateInfo>

DTML will call the updateInfo script. You don't have to specify if the script is implemented in Perl, Python, or any other language (you can also call other DTML objects and SQL Methods this way).
If the updateInfo script requires parameters, you must either choose a name for the DTML namespace binding (see Binding Variables below) so that the parameters will be looked up in the namespace, or you must pass the parameters in an expression, like this:

        <dtml-call expr="updateInfo(color='brown', pattern='spotted')">

Calling scripts from Python and Perl works the same way, except that you must always pass script parameters when you call a script from Python or Perl. For example here's how you might call the updateInfo script from Python:

        context.updateInfo(color='brown', 
                           pattern='spotted')

From Perl you could do the same thing using standard Perl semantics for calling scripts:

        $self->updateInfo(color => 'brown', 
                          pattern => 'spotted');      

Each scripting language has a different way of writing a script call, but you don't have to know what language is used in the script you are calling. Effectively Zope objects can have scripts implemented in several different languages. But when you call a script you don't have to know how it's implemented, you just need to pass the appropriate parameters.
Zope locates the scripts you call using acquisition the same way it does when calling scripts from the web. Returning to our hippo feeding example of the last section, let's see how to vaccinate a hippo from Python and Perl. Figure 8-2 shows a slightly updated object hierarchy that contains two scripts, vaccinateHippo.py and vaccinateHippo.pl.

Figure 8-2 A collection of objects and scripts
Suppose vaccinateHippo.py is a Python script. Here's how you call the vaccinate script on the hippo object from Python:

        context.Vet.LargeAnimals.hippo.vaccinate()

In other words you simply access the object using the same acquisition path as you would use if calling it from the web. Likewise in Perl you could say:

        $self->Vet->LargeAnimals->hippo->vaccinate();

Using scripts from other scripts is very similar to calling scripts from the web. The semantics differ slightly but the same acquisition rules apply. Later on in this chapter, you'll see more examples of how scripts in both Perl and Python work.

Passing Parameters to Scripts

All scripts can be passed parameters. A parameter gives a script more information about what to do. When you call a script from the web, Zope will try to find the script's parameters in the web request and pass them to your script. For example if you have a script with parameters dolphin and REQUEST Zope will look for dolphin in the web request, and will pass the request itself as the REQUEST parameter. In practical terms this means that it is easy to do form processing in your script. For example here is a form:

        <form action="actionScript">
        Name <input type="text" name="name"><br>
        Age <input type="text" name="age:int"><br>
        <input type="submit">
        </form>

You can easily process this form with a script named actionScript that includes name and age in its parameter list:

        ## Script (Python) "actionScript"
        ##parameters=name, age
        ##
        "Process form"
        context.processName(name)
        context.processAge(age)
        return context.responseMessage()

There's no need to process the form manually to extract values from it. Form elements are passed as strings, or lists of strings in the case of check boxes, and multiple-select input.
In addition to form variables, you can specify any request variables as script parameters. For example, to get access to the request and response objects just include REQUEST and RESPONSE in your list of parameters. Request variables are detailed more fully in Appendix B.
One thing to note is that the context variable refers to the object that your script is called on. This works similarly in Perl-based Scripts, for example:

        my $self = shift;
        $self->processName($name);
        $self->processAge($age);
        return $context->responseMessage();

In the Python version of the example, there is a subtle problem. You are probably expecting an integer rather than a string for age. You could manually convert the string to an integer using the Python int built-in:

        age=int(age) # covert a string to an integer

But this manual conversion may be inconvenient. Zope provides a way for you to specify form input types in the form, rather than in the processing script. Instead of converting the age variable to an integer in the processing script, you can indicate that it is an integer in the form:

        Age <input type="text" name="age:int">

The :int appended to the form input name tells Zope to automatically convert the form input to an integer. If the user of your form types something that can't be converted to an integer (such as "22 going on 23") then Zope will raise an exception as shown in Figure 8-3.

Figure 8-3 Parameter conversion error
It's handy to have Zope catch conversion errors, but you may not like Zope's error messages. You should avoid using Zope's converters if you want to provide your own error messages.
Zope can perform many parameter conversions. Here is a list of Zope's basic parameter converters.

boolean

Converts a variable to true or false. Variables that are 0, None, an empty string, or an empty sequence are false, all others are true.

int

Converts a variable to an integer.

long

Converts a variable to a long integer.

float

Converts a variable to a floating point number.

string

Converts a variable to a string. Most variables are strings already so this converter is seldom used.

text

Converts a variable to a string with normalized line breaks. Different browsers on various platforms encode line endings differently, so this script makes sure the line endings are consistent, regardless of how they were encoded by the browser.

list

Converts a variable to a Python list.

tuple

Converts a variable to a Python tuple. A tuple is like a list, but cannot be modified.

tokens

Converts a string to a list by breaking it on white spaces.

lines

Converts a string to a list by breaking it on new lines.

date

Converts a string to a DateTime object. The formats accepted are fairly flexible, for example 10/16/2000, 12:01:13 pm.

required

Raises an exception if the variable is not present.

ignore_empty

Excludes the variable from the request if the variable is an empty string.

These converters all work in more or less the same way to coerce a string form variable into a specific type. You may recognize these converters from Chapter 3, "Using Basic Zope Objects", where we discussed properties. These converters are used by Zope's property facility to convert properties to the right type.
The list and tuple converters can be used in combination with other converters. This allows you to apply additional converters to each element of the list or tuple. Consider this form:

        <form action="processTimes"> 

        <p>I would prefer not to be disturbed at the following
        times:</p>

        <input type="checkbox" name="disturb_times:list:date"
        value="12:00 AM"> Midnight<br>

        <input type="checkbox" name="disturb_times:list:date"
        value="01:00 AM"> 1:00 AM<br>

        <input type="checkbox" name="disturb_times:list:date"
        value="02:00 AM"> 2:00 AM<br>

        <input type="checkbox" name="disturb_times:list:date"
        value="03:00 AM"> 3:00 AM<br>

        <input type="checkbox" name="disturb_times:list:date"
        value="04:00 AM"> 4:00 AM<br>

        <input type="submit">
        </form>

By using the list and date converters together Zope will convert each selected time to a date and then combine all selected dates into a list named disturb_times.
A more complex type of form conversion is to convert a series of inputs into records. Records are structures that have attributes. Using records you can combine a number of form inputs into one variable with attributes. The available record converters are:

record

Converts a variable to a record attribute.

records

Converts a variable to a record attribute in a list of records.

default

Provides a default value for a record attribute if the variable is empty.

ignore_empty

Skips a record attribute if the variable is empty.

Here are some examples of how these converters are used:

        <form action="processPerson">

        First Name <input type="text" name="person.fname:record"><br>
        Last Name <input type="text" name="person.lname:record"><br>
        Age <input type="text" name="person.age:record:int"><br>

        <input type="submit">
        </form>

This form will call the processPerson script with one parameter, person. The person variable will have fname, lname and age attributes. Here's an example of how you might use the person variable in your processPerson script:

        ## Script (Python) "processPerson"
        ##parameters=person
        ##
        " process a person record "
        full_name="%s %s" % (person.fname, person.lname)
        if person.age < 21:
            return "Sorry, %s. You are not old enough to adopt an aardvark." % full_name
        return "Thanks, %s. Your aardvark is on its way." % full_name

The records converter works like the record converter except that it produces a list of records, rather than just one. Here's an example form:

        <form action="processPeople">

        <p>Please, enter information about one or more of your next of
        kin.</p>

        <p>First Name <input type="text" name="people.fname:records">
        Last Name <input type="text" name="people.lname:records"></p>

        <p>First Name <input type="text" name="people.fname:records">
        Last Name <input type="text" name="people.lname:records"></p>

        <p>First Name <input type="text" name="people.fname:records">
        Last Name <input type="text" name="people.lname:records"></p>

        <input type="submit">
        </form>    

This form will call the processPeople script with a variable called people that is a list of records. Each record will have fname and lname attributes.
Another useful parameter conversion uses form variables to rewrite the action of the form. This allows you to submit a form to different scripts depending on how the form is filled out. This is most useful in the case of a form with multiple submit buttons. Zope's action converters are:

action

Changes the action of the form. This is mostly useful in the case where you have multiple submit buttons on one form. Each button can be assigned to a script that gets called when that button is clicked to submit the form.

default_action

Changes the action script of the form when no other method converter is found.

Here's an example form that uses action converters:

        <form action="">

        <p>Select one or more employees</p>

        <input type="checkbox" name="employees:list" value="Larry"> Larry<br>
        <input type="checkbox" name="employees:list" value="Simon"> Simon<br>
        <input type="checkbox" name="employees:list" value="Rene"> Rene<br>

        <input type="submit" name="fireEmployees:action"
        value="Fire!"><br>

        <input type="submit" name="promoteEmployees:action"
        value="Promote!">

        </form>

This form will call either the fireEmployees or the promoteEmployees script depending on which of the two submit buttons is used. Notice also how it builds a list of employees with the list converter. Form converters can be very useful when designing Zope applications.

Script Security

All scripts that can be edited through the web are subject to Zope's standard security policies. The only scripts that are not subject to these security restrictions are scripts that must be edited through the filesystem. These unrestricted scripts include Python and Perl External Methods.
Chapter 7, "Users and Security" covers security in more detail. You should consult the Roles of Executable Objects and Proxy Roles sections for more information on how scripts are restricted by Zope security constraints.

The Zope API

One of the main reasons to script Zope is to get convenient access to the Zope API (Application Programmer Interface). The Zope API describes built-in actions that can be called on Zope objects. You can examine the Zope API in the help system, as shown in Figure 8-4.

Figure 8-4 Zope API Documentation
Suppose you'd like to have a script that takes a file you upload from a form and creates a Zope File object in a folder. To do this you need to know a number of Zope API actions. It's easy enough to read files in Python or Perl, but once you have the file you need to know what actions to call to create a new File object in a Folder.
There are many other things that you might like to script using the Zope API. Any management task that you can perform through the web can be scripted using the Zope API. This includes creating, modifying and deleting Zope objects. You can even perform maintenance tasks, like restarting Zope and packing the Zope database.
The Zope API is documented in Appendix B, "API Reference" as well as in the Zope online help. The API documentation shows you which classes inherit from which other classes. For example Folder inherits from ObjectManager. This means that Folder objects have all the actions listed in the ObjectManager section of the API reference.

Using Python-based Scripts

Earlier in this chapter you saw some examples of scripts. Now let's take a look at scripts in more detail.

The Python Language

Python is a high-level, object oriented scripting language. Most of Zope is written in Python. Many folks like Python because of its clarity, simplicity and ability to scale to large projects.
There are many resources available for learning Python. The python.org web site has lots of Python documentation including a tutorial by Python's Creator, Guido van Rossum.
Python comes with a rich set of modules and packages. You can find out more about the Python standard library at the python.org web site.
Another highly respected source for reference material is Python Essential Reference by David Beazley published by New Riders.

Creating Python-based Scripts

To create a Python-based Script choose Script (Python) from the Product add list. Name the script hello, and click the Add and Edit button. You should now see the Edit view of your script as shown in Figure 8-5.

Figure 8-5 Script editing view
This screen allows you to control the parameters and body of your script. You can enter your script's parameters in the parameter list field. Type the body of your script in the text area at the bottom of the screen.
Enter name="World" into the parameter list field, and type:

        return "Hello %s." % name

in the body of the script. This is equivalent to this in standard Python syntax:

        def hello(name="World"):
            return "Hello %s." % name

You can now test this script by going to the Test tab as shown in Figure 8-6.

Figure 8-6 Testing a Script
Leave the name field blank and click the Run Script button. Zope should return "Hello World." Now go back and try entering your name in the Value field and click the Run Script button. Zope should now say hello to you.
Since scripts are called on Zope objects, you can get access to Zope objects via the context variable. For example, this script returns the number of objects contained by a given Zope object:

        ## Script (Python) "numberOfObjects
        ##
        return len(context.objectIds())

The script calls context.objectIds() to find out the number of contained objects. When you call this script on a given Zope object, the context variable is bound to the context object. So if you called this script by visiting the URL FolderA/FolderB/numberOfObjects the context parameter would refer to the FolderB object.
When writing your logic in Python you'll typically want to query Zope objects, call other scripts and return reports. For example, suppose you want to implement a simple workflow system in which various Zope objects are tagged with properties that indicate their status. You might want to produce reports that summarize which objects are in which state. You can use Python to query objects and test their properties. For example, here is a script named objectsForStatus with one parameter, status:

        ## Script (Python) "objectsForStatus"
        ##parameters=status
        ##
        """
        Returns all sub-objects that have a given status
        property.
        """
        results=[]
        for object in context.objectValues():
            if object.getProperty('status') == status:
                results.append(object)
        return results

This script loops through an object's sub-objects and returns all the sub-objects that have a status property with a given value.
You could then use this script from DTML to email reports. For example:

        <dtml-sendmail>
        To: <dtml-var ResponsiblePerson>
        Subject: Pending Objects

        These objects are pending and need attention.

        <dtml-in expr="objectsForStatus('Pending')">
        <dtml-var title_or_id> (<dtml-var absolute_url>)
        </dtml-in>
        </dtml-sendmail>

This example shows how you can use DTML for presentation or report formatting, while Python handles the logic. This is a very important pattern, that you'll see over and over in Zope.

String Processing

One common use for scripts is to do string processing. Python has a number of standard modules for string processing. You cannot do regular expression processing from Python-based Scripts, but you do have access to the string module. You have access to the string module from DTML as well, but it is much easier to use from Python. Suppose you want to change all the occurrences of a given word in a DTML Document. Here's a script, replaceWord, that accepts two arguments, word and replacement. This will change all the occurrences of a given word in a DTML Document:

        ## Script (Python) "replaceWord"
        ##parameters=word, replacement
        ##
        """
        Replaces all the occurrences of a word with a
        replacement word in the source text of a DTML
        Document. Call this script on a DTML Document to use
        it. 

        Note: you'll need permission to edit a document to
        call this script on the document.
        """
        import string
        text=context.document_src()
        text=string.replace(text, word, replacement)
        context.manage_edit(text, context.title)

You can call this script from the web on a DTML Document to change the source of the document. For example, the URL Swamp/replaceWord?word=Alligator&replacement=Crocodile would call the replaceWord script on a document named Swamp and would replace all occurrences of the word Alligator with Crocodile.
The string module that you can access via scripts does not have all the features available in the standard Python string module. These limitations are imposed for security reasons. See Appendix A for more information on the string module.
One thing that you might be tempted to do with scripts is to use Python to search for objects that contain a given word in their text or as a property. You can do this, but Zope has a much better facility for this kind of work, the Catalog. See Chapter 11, "Searching and Categorizing Content" for more information on searching with Catalogs.

Doing Math

Another common use of scripts is to perform mathematical calculations which would be unwieldy from DTML. The math and random modules give you access from Python to many math functions. These modules are standard Python services as described on the Python.org web site.

math

Mathematical functions such as sin and cos.

random

Pseudo random number generation functions.

One interesting function of the random module is the choice function that returns a random selection from a sequence of objects. Here's an example of how to use this function in a script called randomImage:

        ## Script (Python) "randomImage"
        ##
        """
        When called on a Folder that contains Image objects this
        script returns a random image.
        """
        import random
        return random.choice(context.objectValues('Image'))

Suppose you had a Folder named Images that contained a number of images. You could display a random image from the folder in DTML like so:

        <dtml-with Images>
          <dtml-var randomImage>
        </dtml-with>

This DTML calls the randomImage script on the Images folder. The result is a HTML IMG tag that references a random image in the Images Folder.

Binding Variables

A set of special variables is created whenever a Python-based Script is called. These variables, defined on the Bindings view, are used by your script to access other Zope objects and scripts.
By default, the names of these binding variables are set to reasonable values and you should not need to change them. They are explained here so that you know how each special variable works, and how you can use these variables in your scripts.

Context

The Context binding defaults to the name context. This variable refers to the object that the script is called on.

Container

The Container binding defaults to the name container. This variable refers to the folder that the script is defined in.

Script

The Script binding defaults to the name script. This variable refers to the script object itself.

Namespace

The Namespace binding is left blank by default. This is an advanced variable that you will not need for any of the examples in this book. If your script is called from a DTML Method, and you have chosen a name for this binding, then the named variable contains the DTML namespace explained in Chapter 8, "Variables and Advanced DTML". Also, if this binding is set, the script will search for its parameters in the DTML namespace when called from DTML without explicitly passing any arguments.

Subpath

The Subpath binding defaults to the name traverse_subpath. This is an advanced variable that you will not need for any of the examples in this book. If your script is traversed, meaning that other path elements follow it in a URL, then those path elements are placed in a list, from left to right, in this variable.

If you edit your scripts via FTP, you'll notice that these bindings are listed in comments at the top of your script files. For example:

        ## Script (Python) "example"
        ##bind container=container
        ##bind context=context
        ##bind namespace=
        ##bind script=script
        ##bind subpath=traverse_subpath
        ##parameters=name, age
        ##title=
        ##
        return "Hello %s you are %d years old." % (name, age)

You can change your script's bindings by changing these comments and then uploading your script.

Print Statement Support

Python-based Scripts have a special facility to help you print information. Normally printed data is sent to standard output and is displayed on the console. This is not practical for a server application like Zope since most of the time you do not have access to the server's console. Scripts allow you to use print anyway and to retrieve what you printed with the special variable printed. For example:

        ## Script (Python) "printExample"
        ##
        for word in ('Zope', 'on', 'a', 'rope'):
            print word
        return printed

This script will return:

        Zope
        on
        a
        rope

The reason that there is a line break in between each word is that Python adds a new line after every string that is printed.
You might want to use the print statement to perform simple debugging in your scripts. For more complex output control you probably should manage things yourself by accumulating data, modifying it and returning it manually rather than relying on the print statement.

Security Restrictions

Scripts are restricted in order to limit their ability to do harm. What could be harmful? In general, scripts keep you from accessing private Zope objects, making harmful changes to Zope objects, hurting the Zope process itself, and accessing the server Zope is running on. These restrictions are implemented through a collection of limits on what your scripts can do.

Loop limits

Scripts cannot create infinite loops. If your script loops a very large number of times Zope will raise an error. This restriction covers all kinds of loops including for and while loops. The reason for this restriction is to limit your ability to hang Zope by creating an infinite loop.

Import limits

Scripts cannot import arbitrary packages and modules. You are limited to importing the Products.PythonScripts.standard utility module, the AccessControl module, those modules available via DTML (string, random, math, sequence), and modules which have been specifically made available to scripts by product authors. See Appendix B, "API Reference" for more information on these modules. If you want to be able to import any Python module, use an External Method, as described later in the chapter.

Access limits

You are restricted by standard Zope security policies when accessing objects. In other words the user executing the script is checked for authorization when accessing objects. As with all executable objects you can modify the effective roles a user has when calling a script using Proxy Roles (see Chapter 7, "Users and Security", for more information.) In addition, you cannot access objects whose names begin with underscore, since Zope considers these objects to be private.

Writing limits

In general you cannot change Zope object attributes using scripts. You should call scripts on Zope objects to change them, rather than directly changing instance attributes.

Despite these limits, a determined user could use large amounts of CPU time and memory using Python-based Scripts. So malicious scripts could constitute a kind of denial of service attack by using lots of resources. These are difficult problems to solve and DTML suffers from the same potential for abuse. As with DTML, you probably shouldn't grant access to scripts to untrusted people.

Built-in Functions

Python-based Scripts give you a slightly different menu of built-ins than you find in normal Python. Most of the changes are designed to keep you from performing unsafe actions. For example, the open function is not available, which keeps you from being able to access the filesystem. To partially make up for some missing built-ins a few extra functions are available.
These restricted built-ins work the same as standard Python built-ins: None, abs, apply, callable, chr, cmp, complex, delattr, divmod, filter, float, getattr, hash, hex, int, isinstance, issubclass, list, len, long, map, max, min, oct, ord, repr, round, setattr, str, tuple. For more information on what these built-ins do, see the online Python Documentation.
The range and pow functions are available and work the same way they do in standard Python; however, they are limited to keep them from generating very large numbers and sequences. This limitation helps protect against denial of service attacks as described previously.
In addition, these DTML utility functions are available: DateTime, and test. See Appendix A, "DTML Reference" for more information on these functions.
Finally to make up for the lack of a type function, there is a same_type function that compares the type of two or more objects, returning true if they are of the same type. So instead of saying:

        if type(foo) == type([]):
            return "foo is a list"

to check if foo is a list, you would instead use the same_type function to check this:

        if same_type(foo, []):
            return "foo is a list"

Now let's take a look at External Methods which provide more power and less restrictions than Python-based Scripts.

Using External Methods

Sometimes the security constraints imposed by scripts get in your way. For example, you might want to read files from disk, or access the network, or use some advanced libraries for things like regular expressions or image processing. In these cases you'll want to use External Methods.
To create and edit External Methods you need access to the filesystem. This makes editing these scripts more cumbersome since you can't edit them right in your web browser. However requiring access to the server's filesystem provides an important security control. If a user has access to a servers filesystem they already have the ability to harm Zope. So by requiring that unrestricted scripts be edited on the filesystem Zope ensures that only people who are already trusted have access.
Unrestricted scripts are created and edited in files on the Zope server in the Extensions directory. This directory is located in the top-level Zope directory. Alternately you can create and edit unrestricted scripts in an Extensions directory inside an installed Zope product directory.
Create a file named Example.py in the Zope Extensions directory on your server. In the Example.py file, enter the following code:

      def hello(name="World"):
          return "Hello %s." % name 

You've created a Python function in a Python module. Now let's use this function in the External Method.
You manage External Methods the same way you manage restricted scripts with the exception that you cannot edit the script itself through the web. Instead of editing code you must tell Zope where to find your code on the filesystem. You do this by specifying the name of your Python file and the name of the function within the module.
To create an External Method choose External Method from the product add list. You will be taken to an add form where you must provide an id. Type "hello" into the Id field and "hello" in the Function name field and "Example" in the Module name field and click the Add button. You should now see a new External Method object in your folder. Click on it. You should be taken to the Properties view of your new External Method as shown in Figure 8-7.

Figure 8-7 External Method Properties view
Now test your new script by going to the Test view. You should see a greeting. You can pass different names to the script by specifying them in the URL. For example, hello?name=Spanish+Inquisition.
This example is exactly the same as the hello world example that you saw for using scripts. In fact for simple string processing tasks like this restricted scripts offer a better solution since they are easier to work with.
The main reasons to use an unrestricted script are to access the filesystem or network or to use Python packages that are not available to restricted scripts.
Here's an example External Method that uses the Python Imaging Library (PIL) to create a thumbnail version of an existing Image object in a Folder. Enter the following code in a file named Thumbnail.py in the Extensions directory:

      def makeThumbnail(self, original_id, size=200):
          """
          Makes a thumbnail image given an image Id when called on a Zope
          folder.

          The thumbnail is a Zope image object that is a small JPG
          representation of the original image. The thumbnail has a
          'original_id' property set to the id of the full size image
          object.
          """

          from PIL import Image
          from StringIO import StringIO
          import os.path

          # create a thumbnail image file
          original_image=getattr(self, original_id)
          original_file=StringIO(str(original_image.data))
          image=Image.open(original_file)
          image=image.convert('RGB')
          image.thumbnail((size,size))
          thumbnail_file=StringIO()
          image.save(thumbnail_file, "JPEG") 
          thumbnail_file.seek(0)

          # create an id for the thumbnail
          path, ext=os.path.splitext(original_id)
          thumbnail_id=path + '.thumb.jpg'

          # if there's and old thumbnail, delete it
          if thumbnail_id in self.objectIds():
              self.manage_delObjects([thumbnail_id])

          # create the Zope image object
          self.manage_addProduct['OFSP'].manage_addImage(thumbnail_id,
                                                         thumbnail_file,
                                                         'thumbnail image')
          thumbnail_image=getattr(self, thumbnail_id)

          # set the 'originial_id' property
          thumbnail_image.manage_addProperty('original_id', original_id, 'string')

You must have PIL installed for this example to work. See the PythonWorks website for more information on PIL. To use this code create an External Method named makeThumbnail that uses the makeThumbnail function in the Thumbnail module.
Now you have a method that will create a thumbnail image. You can call it on a Folder with a URL like ImageFolder/makeThumbnail?original_id=Horse.gif This would create a thumbnail image named Horse.thumb.jpg.
You can use a script to loop through all the images in a folder and create thumbnail images for them. Create a script named makeThumbnails:

      ## Script (Python) "makeThumbnails"
      ##
      for image_id in context.objectIds('Image'):
          context.makeThumbnail(image_id)

This will loop through all the images in a folder and create a thumbnail for each one.
Now call this script on a folder with images in it. It will create a thumbnail image for each contained image. Try calling the makeThumbnails script on the folder again and you'll notice it created thumbnails of your thumbnails. This is no good. You need to change the makeThumbnails script to recognize existing thumbnail images and not make thumbnails of them. Since all thumbnail images have an original_id property you can check for that property as a way of distinguishing between thumbnails and normal images:

      ## Script (Python) "makeThumbnails"
      ##
      for image in context.objectValues('Image'):
          if not image.hasProperty('original_id'):
              context.makeThumbnail(image.getId())

Delete all the thumbnail images in your folder and try calling your updated makeThumbnails script on the folder. It seems to work correctly now.
Now with a little DTML you can glue your script and External Method together. Create a DTML Method called displayThumbnails:

      <dtml-var standard_html_header>

      <dtml-if updateThumbnails>
        <dtml-call makeThumbnails>
      </dtml-if>

      <h2>Thumbnails</h2>

      <table><tr valign="top">

      <dtml-in expr="objectValues('Image')">
        <dtml-if original_id>
          <td>
            <a href="&dtml-original_id;"><dtml-var sequence-item></a><br>
            <dtml-var original_id>
          </td> 
        </dtml-if>
      </dtml-in>

      </tr></table>

      <form>
      <input type="submit" name="updateThumbnails" value="Update Thumbnails">
      </form>

      <dtml-var standard_html_footer>

When you call this DTML Method on a folder it will loop through all the images in the folder and display all the thumbnail images and link them to the originals as shown in Figure 8-8.

Figure 8-8 Displaying thumbnail images
This DTML Method also includes a form that allows you to update the thumbnail images. If you add, delete or change the images in your folder you can use this form to update your thumbnails.
This example shows how to use scripts, External Methods and DTML together. Python takes care of the logic while the DTML handles presentation. Your External Methods handle external packages while your scripts do simple processing of Zope objects.

Processing XML with External Methods

You can use External Methods to do darn near anything. One interesting thing that you can do is to communicate using XML. You can generate and process XML with External Methods.
Zope already understands some kinds of XML messages such as XML-RPC and WebDAV. As you create web applications that communicate with other systems you may want to have the ability to receive XML messages. You can receive XML a number of ways: you can read XML files from the file system or over the network, or you can define scripts that take XML arguments which can be called by remote systems.
Once you have received an XML message you must process the XML to find out what it means and how to act on it. Let's take a quick look at how you might parse XML manually using Python. Suppose you want to connect your web application to a Jabber chat server. You might want to allow users to message you and receive dynamic responses based on the status of your web application. For example suppose you want to allow users to check the status of animals using instant messaging. Your application should respond to XML instant messages like this:

        <message to="cage_monitor@zopezoo.org" from="user@host.com">
          <body>monkey food status</body>
        </message>

You could scan the body of the message for commands, call a script and return responses like this:

        <message to="user@host.com" from="cage_monitor@zopezoo.org">
          <body>Monkeys were last fed at 3:15</body>
        </message>

Here is a sketch of how you could implement this XML messaging facility in your web application using an External Method:

        # Uses Python 2.x standard xml processing packages.  See
        # http://www.python.org/doc/current/lib/module-xml.sax.html for
        # information about Python's SAX (Simple API for XML) support If
        # you are using Python 1.5.2 you can get the PyXML package. See
        # http://pyxml.sourceforge.net for more information about PyXML.

        from xml.sax import parseString
        from xml.sax.handler import ContentHandler

        class MessageHandler(ContentHandler):
            """
            SAX message handler class

            Extracts a message's to, from, and body
            """

            inbody=0
            body=""

            def startElement(self, name, attrs):
                if name=="message":
                    self.recipient=attrs['to']
                    self.sender=attrs['from']
                elif name=="body":
                    self.inbody=1

            def endElement(self, name):
                if name=="body":
                    self.inbody=0

            def characters(self, content):
                if self.inbody:
                    self.body=self.body + content

        def receiveMessage(self, message):
            """
            Called by a Jabber server
            """
            handler=MessageHandler()
            parseString(message, handler)

            # call a script that returns a response string
            # given a message body string
            response_body=self.getResponse(handler.body)

            # create a response XML message
            response_message="""
              <message to="%s" from="%s">
                <body>%s</body>
              </message>""" % (handler.sender, handler.recipient, response_body)

            # return it to the server
            return response_message

The receiveMessage External Method uses Python's SAX (Simple API for XML) package to parse the XML message. The MessageHandler class receives callbacks as Python parses the message. The handler saves information its interested in. The External Method uses the handler class by creating an instance of it, and passing it to the parseString function. It then figures out a response message by calling getResponse with the message body. The getResponse script (which is not shown here) presumably scans the body for commands, queries the web applications state and returns some response. The receiveMessage method then creates an XML message using response and the sender information and returns it.
The remote server would use this External Method by calling the receiveMessage method using the standard HTTP POST command. Voila, you've implemented a custom XML chat server that runs over HTTP.

External Method Gotchas

While you are essentially unrestricted in what you can do in an External Method, there are still some things that are hard to do.
While your Python code can do as it pleases if you want to work with the Zope framework you need to respect its rules. While programming with the Zope framework is too advanced a topic to cover here, there are a few things that should be aware of.
Problems can occur if you hand instances of your own classes to Zope and expect them to work like Zope objects. For example, you cannot define a class in an External Method script file and assign it as an attribute of a Zope object. This causes problems with Zope's persistence machinery. You also cannot easily hand instances of your own classes over to DTML or scripts. The issue here is that your instances won't have Zope security information. You can define and use your own classes and instances to your heart's delight, just don't expect Zope to use them directly. Limit yourself to returning simple Python structures like strings, dictionaries and lists or Zope objects.

Using Perl-based Scripts

Perl-based Scripts allow you to script Zope in Perl. If you love Perl and don't want to learn Python to use Zope, these scripts are for you. Using Perl-based Scripts you can use all your favorite Perl modules and treat Zope like a collection of Perl objects.

The Perl Language

Perl is a high-level scripting language like Python. From a broad perspective, Perl and Python are very similar languages, they have similar primitive data constructs and employ similar programming constructs.
Perl is a popular language for Internet scripting. In the early days of CGI scripting, Perl and CGI were practically synonymous. Perl continues to be the dominant Internet scripting language.
Perl has a very rich collection of modules for tackling almost any computing task. CPAN (Comprehensive Perl Archive Network) is the authoritative guide to Perl resources.
Perl-based Zope scripts are available for download from ActiveState. Perl-based scripts require you to have Perl installed, and a few other packages, and how to install these things is beyond the scope of this book. See the documentation that comes with Perl-based scripts from the above URL. There is also more information provided by Andy McKay available on Zope.org.

Creating Perl-based Scripts

Perl-based Scripts are quite similar to Python-based Scripts. Both have access to Zope objects and are called in similar ways. Here's the Perl hello world program:

        my $name=shift;
        return "Hello $name.";

Let's take a look at a more complex example script by Monty Taylor. It uses the LWP::UserAgent package to retrieve the URL of the daily Dilbert comic from the network. Create a Perl-based Script named get_dilbert_url with this code:

        use LWP::UserAgent;

        my $ua = LWP::UserAgent->new;

        # retrieve the Dilbert page
        my $request = HTTP::Request->new('GET','http://www.dilbert.com');
        my $response = $ua->request($request);

        # look for the image URL in the HTML
        my $content = $response->content;
        $content =~ m,(/comics/dilbert/archive/images/[^"]*),s;

        # return the URL
        return $content        

You can display the daily Dilbert comic by calling this script from DTML by calling the script inside an HTML IMG tag:

        <img src="&dtml-get_dilbert_url;">

However there is a problem with this code. Each time you display the cartoon, Zope has to make a network connection. This is inefficient and wasteful. You'd do much better to only figure out the Dilbert URL once a day.
Here's a script cached_dilbert_url that improves the situation by keeping track of when it last fetched the Dilbert URL with a dilbert_url_date property:

        my $context=shift;
        my $date=$context->getProperty('dilbert_url_date');

        if ($date==null or $now-$date > 1){
            my $url=$context->get_dilbert_url();
            $context->manage_changeProperties(
              dilbert_url => $url
              dilbert_url_time => $now
            );
        }
        return $context->getProperty('dilbert_url');

This script uses two properties, dilbert_url and dilbert_url_date. If the URL gets too old, a new one is fetched. You can use this script from DTML just like the original script:

        <img src="&dtml-cached_dilbert_url;">

You can use Perl and DTML together to control your logic and your presentation.

Perl-based Script Security

Like DTML and Python-based Scripts, Perl-based Scripts constrain you in the Zope security system from doing anything that you are not allowed to do. Script security is similar in both languages, but there are some Perl specific constraints.
First, the security system does not allow you to eval an expression in Perl. For example, consider this script:

        my $context = shift;
        my $input = shift;

        eval $input

This code takes an argument and evaluates it in Perl. This means you could call this script from, say an HTML form, and evaluate the contents of one of the form elements. This is not allowed since the form element could contain malicious code.
Perl-based Scripts also cannot assign new variables to any object other than local variables that you declare with my.

DTML versus Python versus Perl

Zope gives you many ways to script. For small scripting tasks the choice of Python, Perl or DTML probably doesn't make a big difference. For larger, logic-oriented tasks you should use Python or Perl. You should choose the language you are most comfortable with. Of course, your boss may want to have some say in the matter too.
Just for comparison sake here is a simple script suggested by Gisle Aas, the author of Perl-based Scripts, in three different languages.
In DTML:

      <dtml-in objectValues>
        <dtml-var getId>: <dtml-var sequence-item>
      </dtml-in>
      done

In Python:

      for item in context.objectValues():
          print "%s: %s" % (item.getId(), item)
      print "done"
      return printed

In Perl:

      my $context = shift;
      my @res;

      for ($context->objectValues()) {
          push(@res, join(": ", $_->getId(), $_));
      }
      join("\n", @res, "done");

Despite the fact that Zope is implemented in Python, it follows the Perl philosophy that there's more than one way to do it.

Remote Scripting and Network Services

Web servers are used to serve content to software clients; usually people using web browser software. The software client can also be another computer that is using your web server to access some kind of service.
Because Zope exposes objects and scripts on the web, it can be used to provide a powerful, well organized, secure web API to other remote network application clients.
There are two common ways to remotely script Zope. The first way is using a simple remote procedure call protocol called XML-RPC. XML-RPC is used to execute a procedure on a remote machine and get a result on the local machine. XML-RPC is designed to be language neutral, and in this chapter you'll see examples in Python, Perl and Java.
The second common way to remotely script Zope is with any HTTP client that can be automated with a script. Many language libraries come with simple scriptable HTTP clients and there are many programs that let you you script HTTP from the command line.

Using XML-RPC

XML-RPC is a simple remote procedure call mechanism that works over HTTP and uses XML to encode information. XML-RPC clients have been implemented for many languages including Python, Perl, Java, JavaScript, and TCL.
In-depth information on XML-RPC can be found at the XML-RPC website.
All Zope scripts that can be called from URLs can be called via XML-RPC. Basically XML-RPC provides a system to marshal arguments to scripts that can be called from the web. As you saw earlier in the chapter Zope provides its own marshaling controls that you can use from HTTP. XML-RPC and Zope's own marshaling accomplish much the same thing. The advantage of XML-RPC marshaling is that it is a reasonably supported standard that also supports marshaling of return values as well as argument values.
Here's a fanciful example that shows you how to remotely script a mass firing of janitors using XML-RPC.
Here's the code in Python:

        import xmlrpclib

        server = xmlrpclib.Server('http://www.zopezoo.org/')
        for employeeID in server.JanitorialDepartment.personnel():
            server.fireEmployee(employee)

In Perl:

        use Frontier::Client;

        $server = Frontier::Client->new(url => "http://www.zopezoo.org/");

        $employees = $server->call("JanitorialDepartment.personnel");
        foreach $employee ( @$employees ) {

          $server->call("fireEmployee",$server->string($employee));

        }

In Java:

        try {
            XmlRpcClient server = new XmlRpcClient("http://www.zopezoo.org/");
            Vector employees = (Vector) server.execute("JanitorialDepartment.personnel");

            int num = employees.size();
            for (int i = 0; i < num; i++) {
                Vector args = new Vector(employees.subList(i, i+1));
                server.execute("fireEmployee", args);
            }

        } catch (XmlRpcException ex) {
            ex.printStackTrace();
        } catch (IOException ioex) {
            ex.printStackTrace();
        }

Actually the above example will probably not run correctly, since you will most likely want to protect the fireEmployee script. This brings up the issue of security with XML-RPC. XML-RPC does not have any security provisions of its own; however, since it runs over HTTP it can leverage existing HTTP security controls. In fact Zope treats an XML-RPC request exactly like a normal HTTP request with respect to security controls. This means that you must provide authentication in your XML-RPC request for Zope to grant you access to protected scripts. The Python client at the time of this writing does not support control of HTTP Authorization headers. However it is a fairly trivial addition. For example, an article on XML.com Internet Scripting: Zope and XML-RPC includes a patch to Python's XML-RPC support showing how to add HTTP authorization headers to your XML-RPC client.

Remote Scripting with HTTP

Any HTTP client can be used for remotely scripting Zope.
On Unix systems you have a number of tools at your disposal for remotely scripting Zope. One simple example is to use wget to call Zope script URLs and use cron to schedule the script calls. For example, suppose you have a Zope script that feeds the lions and you'd like to call it every morning. You can use wget to call the script like so:

        $ wget --spider http://www.zopezope.org/Lions/feed

The spider option tells wget not to save the response as a file. Suppose that your script is protected and requires authorization. You can pass your user name and password with wget to access protected scripts:

        $ wget --spider --http_user=ZooKeeper --http_pass=SecretPhrase http://www.zopezope.org/Lions/feed

Now let's use cron to call this command every morning at 8am. Edit your crontab file with the crontab command:

        $ crontab -e

Then add a line to call wget every day at 8 am:

        0 8 * * * wget -v --spider --http_user=ZooKeeper --http_pass=SecretPhrase http://www.zopezoo.org/Lions/feed

The only difference between using cron and calling wget manually is that you should use the v switch when using cron since you don't care about output of the wget command.
For our final example let's get really perverse. Since networking is built into so many different systems, it's easy to find an unlikely candidate to script Zope. If you had an Internet-enabled toaster you would probably be able to script Zope with it. Let's take Microsoft Word as our example Zope client. All that's necessary is to get Word to agree to tickle a URL.
The easiest way to script Zope with Word is to tell word to open a document and then type a Zope script URL as the file name as shown in Figure 8-9.

Figure 8-9 Calling a URL with Microsoft Word
Word will then load the URL and return the results of calling the Zope script. Despite the fact that Word doesn't let you POST arguments this way, you can pass GET arguments by entering them as part of the URL.
You can even control this behavior using Word's built-in Visual Basic scripting. For example, here's a fragment of Visual Basic that tells Word to open a new document using a Zope script URL:

        Documents.Open FileName:="http://www.zopezoo.org/LionCages/wash?use_soap=1&water_temp=hot" 

You could use Visual Basic to call Zope script URLs in many different ways.
Zope's URL to script call translation is the key to remote scripting. Since you can control Zope so easily with simple URLs you can easy script Zope with almost any network-aware system.

Conclusion

Zope provides scripting with Python and Perl. With scripts you can control Zope objects and glue together your application's logic, data, and presentation. You can also perform serious programming tasks such as image processing and XML parsing.
In the next chapter you'll learn about ZCatalog, Zope's built-in search engine.