Professional Plone 4 Development

P U B L I S H I N G community experience distilled Professional Plone 4 Development Martin Aspeli Chapter No.5 "Developing a Site Strategy"

In this package, you will find: A Biography of the author of the book A preview chapter from the book, Chapter NO.5 "Developing a Site Strategy" A synopsis of the book s content Information on where to buy this book About the Author Martin Aspeli is an experienced Plone consultant and a prolific Plone contributor. He served on the Framework Team for Plone 3.0, and has contributed significantly for many of Plone's features, such as the portlets infrastructure, the content rules engine, the Dexterity content type framework, and the integration of Diazo theming engine into Plone. He is a former leader of the Plone Documentation Team, has written a number of wellreceived tutorials available on plone.org, and is a regular speaker at Plone events and conferences. Martin was recognized in 2008 by Packt Publishing as one of the Most Valuable People in Open Source Content Management Systems for his work on the first edition of Professional Plone Development. By day, Martin works for a 'Big Four' management consultancy, managing web development teams. He is passionate about Open Source, Agile, and high quality software.

Professional Plone 4 Development Plone is a web content management system that features among the top two percent of open source projects and is used by more than 300 solution providers in 57 countries. Its powerful workflow system, outstanding security track record, friendly user interface, elegant development model, and vibrant community makes Plone a popular choice for building content-centric applications. By customizing and extending the base platform, integrators can build unique solutions tailored to specific projects quickly and easily. If you want to create your own web applications and advanced websites using Plone 4, Professional Plone 4 Development is the book you need. The first edition of this book remains one of the most widely read and recommended Plone books. This second edition is completely revised and up-to-date for Plone 4.1, covering new topics such as Dexterity, Diazo, jquery, and z3c.form, as well as improved ways of working with existing technologies such as Buildout, SQLAlchemy, and the Pluggable Authentication Service. It retains the writing style and comprehensive approach that made the first edition so popular. Built around a realistic case study, Professional Plone 4 Development will take you from an understanding of Plone's central concepts, through basic customization, theming, and custom development, to deployment and optimization. What This Book Covers This book is divided into four sections: 1. First, we will introduce Plone and the case study, and learn how to set up a development environment. 2. The second section covers basic customization, including theming a Plone site using Diazo. 3. The third section focuses on custom development building new content types and user interfaces, customizing security, and integrating with the external databases. 4. The final chapters cover deployment and performance optimization.

Let us take a look at each chapter in a bit more detail: Chapter 1, Plone in Context, discusses what Plone is and when it may be an appropriate choice, and introduces the Plone community. Chapter 2, Introduction to the Case Study, introduces the case study that will be used as the basis for the examples throughout the book. Chapter 3, The Development Environment, discusses how to set up a productive development environment with tools for source code management, debugging, and more. Chapter 4, Basics of Customization, discusses the ways in which the Zope application server infrastructure allows us to customize various aspects of Plone. Chapter 5, Developing a Site Strategy, will start the customization of Plone to meet the requirements of our case study by creating a "policy package" to contain configuration and new code. Chapter 6, Security and Workflow, discusses Zope's security model and shows how to create a custom workflow and permission scheme for our application. Chapter 7, Using Add-ons, shows how to safely install Plone add-ons. Chapter 8, Creating a Custom Theme, uses the Diazo theming engine to turn an HTML mock-up of the final site into a fully functional Plone theme. Chapter 9, Nine Core Concepts of Zope Programming, takes a break from the case study to allow us to go into detail of the core concepts that underpin all Zope programming. Chapter 10, Custom Content Types, uses the Dexterity framework to model the case study's specific data requirements as new content types. Chapter 11, Standalone Views and Forms, shows how to render pages that are not the views of content types, and use the z3c.form framework to generate forms automatically from declarative schemata. Chapter 12, Relational Databases, shows how to query and manipulate an external relational database from within Plone. Chapter 13, Users and their Permissions, shows how to manage personal information and create more advanced security schemes. We will also create a simple plugin to allow users to log into a Plone site using their Facebook account. Chapter 14, Dynamic User Interfaces with jquery, shows how to use jquery, the popular JavaScript framework, to create dynamic user interfaces and manage client-side interactions. Chapter 15, Summary and Potential Enhancements, summarizes the work that has gone into the case study and points to some potential future enhancements.

Chapter 16, Zope on the Server, discusses the differences between a Zope instance configured for development and ones configured for use on a server. Chapter 17, Setting up a Production Server, discusses the services that are typically deployed alongside Zope, including web servers, caching proxies, and load balancers. Chapter 18, Authenticating with LDAP and Active Directory, shows how to configure authentication against an organization's existing LDAP or Active Directory repository. Chapter 19, Looking to the Future, briefly considers migrations, before bringing the book to a close.

Developing a Site Strategy In the previous chapter, we learned about ways in which a developer can customize Plone changing the source code of Plone itself. We will now employ some of these techniques as we begin the process of turning an out of the box Plone installation into the site that will become the Optilux Cinemas website. In this chapter, we will: Create the initial version of a policy package that will help us customize Plone to meet the requirements from Chapter 2, Introduction to the Case Study Add a GenericSetup extension profile to this package Write our first tests to prove that our customizations are working as expected Creating a policy package Our policy package is just a package that can be installed as a Plone add-on. We will use a GenericSetup extension profile in this package to turn a standard Plone installation into one that is configured to our client's needs. We could have used a full-site GenericSetup base profile instead, but by using a GenericSetup extension profile we can avoid replicating the majority of the configuration that is done by Plone. We will use ZopeSkel, which we installed in Chapter 3, The Development Environment, to create an initial skeleton for the package, which we will call optilux.policy, adopting the optilux.* namespace for all Optilux-specific packages.

Developing a Site Strategy In your own code, you should of course use a different namespace. It is usually a good idea to base this on the owning organization's name, as we have done here. Note that package names should be all lowercase, without spaces, underscores, or other special characters. If you intend to release your code into the Plone Collective, you can use the collective.* namespace, although other namespaces are allowed too. The plone.* namespace is reserved for packages in the core Plone repository, where the copyright has been transferred to the Plone Foundation. You should normally not use this without first coordinating with the Plone Framework Team. We go into the src/ directory of the buildout we created in Chapter 3, The Development Environment, and run the following command: $../bin/zopeskel plone optilux.policy This uses the plone ZopeSkel template to create a new package called optilux.policy. This will ask us a few questions. We will stick with "easy" mode for now, and answer True when asked whether to register a GenericSetup profile. Note that ZopeSkel will download some packages used by its local command support. This may mean the initial bin/zopeskel command takes a little while to complete, and assumes that we are currently connected to the internet. A local command is a feature of PasteScript, upon which ZopeSkel is built. ZopeSkel registers an addcontent command, which can be used to insert additional snippets of code, such as view registrations or new content types, into the initial skeleton generated by ZopeSkel. We will not use this feature in this book, preferring instead to retain full control over the code we write and avoid the potential pitfalls of code generation. If you wish to use this feature, you will either need to install ZopeSkel and PasteScript into the global Python environment, or add PasteScript to the ${zopeskel:eggs} option in buildout.cfg, so that you get access to the bin/paster command. Run bin/zopeskel --help from the buildout root directory for more information about ZopeSkel and its options. [ 88 ]

Chapter 5 Distribution details Let us now take a closer look at what ZopeSkel has generated for us. We will also consider which files should be added to version control, and which files should be ignored. Item Version control Purpose setup.py Yes Contains instructions for how Setuptools/Distribute (and thus Buildout) should manage the package's distribution. We will make a few modifications to this file later. setup.cfg Yes Contains additional distribution configuration. In this case, ZopeSkel keeps track of which template was used to generate the initial skeleton using this file. *.egg No ZopeSkel downloads a few eggs that are used for its local command support (Paste, PasteScript, and PasteDeploy) into the distribution directory root. If you do not intend to use the local command support, you can delete these. You should not add these to version control. README.txt Yes If you intend to release your package to the public, you should document it here. PyPI requires that this file be present in the root of a distribution. It is also read into the long_description variable in setup.py. PyPI will attempt to render this as restructuredtext markup (see http://docutils. sourceforge.net/rst.html). docs/ Yes Contains additional documentation, including the software license (which should be the GNU General Public License, version 2, for any packages that import directly from any of Plone's GPL-licensed packages) and a change log. [ 89 ]

Developing a Site Strategy Item Version control [ 90 ] Purpose docs/history.txt Yes Used to manage the change log for any package releases, such as, README.txt, this is read into the long_description in setup. py. Again, if you delete this file, you must modify setup.py to not use it. optilux.policy.egg-info/ No Contains Setuptools/ Distribute runtime information generated from setup.py and setup.cfg. This should not be under version control. optilux/ Yes The namespace package. Contains an init.py file with some Setuptools/Distribute boilerplate necessary for namespace packages to work. optilux/policy Yes The root of the package itself. All our Python, ZCML, GenericSetup, and other files will end up in this directory and its subdirectories. If you installed an "omelette" (see Chapter 3, The Development Environment), this directory can also be found under parts/omelette/ optilux/policy. optilux/policy/ init.py Yes A usually empty file used to make this directory a package may also contain a Zope 2 product initialize() function. This is mainly needed for packages that install Archetypes content types, as we will see in Chapter 10, Custom Content types. optilux/policy/configure.zcml Yes The main Zope configuration file for this package. This is automatically loaded by Plone at startup via an entry point in setup.py see below. optilux/policy/tests.py Yes Contains skeleton integration tests. We will replace these with our own tests later in this chapter.

[ 91 ] Chapter 5 Changes to setup.py Before we can progress, we will make a few modifications to setup.py. Our revised file looks similar to the following code, with changes highlighted: from setuptools import setup, find_packages import os version = '2.0' setup(name='optilux.policy', version=version, description="policy package for the Optilux Cinemas project", long_description=open("readme.txt").read() + "\n" + open(os.path.join("docs", "HISTORY.txt")).read(), # Get more strings from # http://pypi.python.org/pypi?%3aaction=list_classifiers classifiers=[ "Framework :: Plone", "Programming Language :: Python", ], keywords='', author='martin Aspeli', author_email='optilude@gmail.com', url='http://optilux-cinemas.com', license='gpl', packages=find_packages(exclude=['ez_setup']), namespace_packages=['optilux'], include_package_data=true, zip_safe=false, install_requires=[ 'setuptools', 'Plone', ], extras_require={ 'test': ['plone.app.testing',] }, entry_points=""" # -*- Entry points: -*- [z3c.autoinclude.plugin] target = plone """, # setup_requires=["pastescript"], # paster_plugins=["zopeskel"], )

Developing a Site Strategy The changes are as follows: 1. We have added an author name, e-mail address, and updated project URL. These are used as metadata if the distribution is ever uploaded to PyPI. For internal projects, they are less important. 2. We have declared an explicit dependency on the Plone distribution, that is, on Plone itself. This ensures that when our package is installed, so is Plone. We will shortly update our main working set to contain only the optilux. policy distribution. This dependency ensures that Plone is installed as part of our application policy. 3. We have then added a [tests] extra, which adds a dependency on plone. app.testing. We will install this extra as part of the following test working set, making plone.app.testing available in the test runner (but not in the Zope runtime). 4. Finally, we have commented out the setup_requires and paster_plugins option s. These are used to support ZopeSkel local commands, which we have decided not to use. The main reason to comment them out is to avoid having Buildout download these additional dependencies into the distribution root directory, saving time, and reducing the number of files in the build. Also note that, unlike distributions downloaded by Buildout in general, there is no "offline" support for these options. Changes to configure.zcml We will also make a minor change to the generated configure.zcml file, removing the line: <five:registerpackage package="." initialize=".initialize" /> This directive is used to register the package as an old-style Zope 2 product. The main reason to do this is to ensure that the initialize() function is called on Zope startup. This may be a useful hook, but most of the time it is superfluous, and requires additional test setup that can make tests more brittle. See Chapter 7, Using Add-ons, for an example of tests that use the installproduct() function to deal with old-style Zope 2 products in test setup. We can also remove the (empty) initialize() function itself from the optilux/policy/ init.py file, effectively leaving the file blank. Do not delete init.py, however, as it is needed to make this directory into a Python package. [ 92 ]

Updating the buildout Before we can use our new distribution, we need to add it to our development buildout. We will consider two scenarios: [ 93 ] Chapter 5 1. The distribution is under version control in a repository module separate to the development buildout itself. This is the recommended approach, as outlined in Chapter 3, The Development Environment. 2. The distribution is not under version control, or is kept inside the version control module of the buildout itself. The example source code that comes with this book is distributed as a simple archive, so it uses this approach. Given the approach we have taken to separating out our buildout configuration into multiple files, we must first update packages.cfg to add the new package. Under the [sources] section, we could add: [sources] optilux.policy = svn https://some-svn-server/optilux.policy/trunk Or, for distributions without a separate version control URL: [sources] optilux.policy = fs optilux.policy We must also update the main and test working sets in the same file: [eggs] main = optilux.policy test = optilux.policy [test] Finally, we must tell Buildout to automatically add this distribution as a develop egg when running the development buildout. This is done near the top of buildout.cfg: auto-checkout = optilux.policy We must rerun buildout to let the changes take effect: $ bin/buildout We can test that the package is now available for import using the zopepy interpreter : $ bin/zopepy >>> from optilux import policy >>>

Developing a Site Strategy The absence of an ImportError tells us that this package will now be known to the Zope instance in the buildout. To be absolutely sure, you can also open the bin/instance script in a text editor (bin/instance-script.py on Windows) and look for a line in the sys.path mangling referencing the package. Working sets and component configuration It is worth deliberating a little more on how Plone and our new policy package are loaded and configured. At build time: 1. Buildout installs the [instance] part, which will generate the bin/instance script. 2. The plone.recipe.zope2instance recipe calculates a working set from its eggs option, which in our buildout references ${eggs:main}. 3. This contains exactly one distribution: optilux.policy. 4. This in turn depends on the Plone distribution (we will add additional dependencies later in the book), which in turn causes Buildout to install all of Plone. Here, we have made a policy decision to depend on a "big" Plone distribution that includes some optional add-ons. We could also have depended on the smaller Products.CMFPlone distribution (which works for Plone 4.0.2 onwards), which includes only the core of Plone, perhaps adding specific dependencies for add-ons we are interested in. When declaring actual dependencies used by distributions that contain reusable code instead of just policy, you should always depend on the packages you import from or otherwise depend on, and no more. That is, if you import from Products.CMFPlone, you should depend on this, and not on the Plone meta-egg (which itself contains no code, but only declares dependencies on other distributions, including Products. CMFPlone). To learn more about the rationale behind the Products. CMFPlone distribution, see http://dev.plone.org/plone/ ticket/10877. [ 94 ]

Chapter 5 At runtime: 1. The bin/instance script starts Zope. 2. Zope loads the site.zcml file (parts/instance/etc/site.zcml) as part of its startup process. 3. This automatically includes the ZCML configuration for packages in the Products.* namespace, including Products.CMFPlone, Plone's main package. 4. Plone uses z3c.autoinclude to automatically load the ZCML configuration of packages that opt into this using the z3c.autoinclude.plugin entry point target = plone. 5. The optilux.policy distribution contains such an entry point, so it will be configured, along with any packages or files it explicitly includes from its own configure.zcml file. (We will see an example of the including additional dependencies in Chapter 7, Using Add-ons.) Creating an extension profile Let us now register an extension profile for the policy package. ZopeSkel has already done some of the work for us. In configure.zcml, we have: <configure xmlns="http://namespaces.zope.org/zope" xmlns:five="http://namespaces.zope.org/five" xmlns:i18n="http://namespaces.zope.org/i18n" xmlns:genericsetup="http://namespaces.zope.org/genericsetup" i18n_domain="optilux.policy"> <genericsetup:registerprofile name="default" title="optilux Site Policy" directory="profiles/default" description="turn a Plone site into the Optilux site." provides="products.genericsetup.interfaces.extension" /> <!-- -*- extra stuff goes here -*- --> </configure> [ 95 ]

Developing a Site Strategy The XML standard requires that all namespaces be declared using the xmlns: syntax seen on the first few lines of this file. A common mistake is to use a directive such as <genericsetup:registerprofile />, but forget to declare the genericsetup namespace. This will result in an "unknown directive" error. The <genericsetup:registerprofile /> stanza registers a new profile. The title and description (which we have edited from their generated defaults) will be shown to the user when activating the policy package. The name is almost always default, unless the package contains multiple profiles. The full profile name as known to GenericSetup includes the package name, so in this case, it will be profileoptilux.policy:default. The profile- prefix indicates that this is an (extension) profile. GenericSetup snapshots have names starting with snapshot-. The directory argument tells GenericSetup where to look for the XML files which the various import handles will read, relative to the package. By convention, this is profiles/default for the primary profile. We then create the profile directory (src/optilux.policy/optilux/policy/ profiles/default), and add a metadata.xml file inside it: <metadata> <version>1</version> <dependencies> </dependencies> </metadata> This defines the profile version, which should always start at 1. It will stay that way until we need to worry about upgrades (see Chapter 16, Zope on the Server). We have also added an empty dependencies list, as our package currently has no dependencies. We will see how to install dependencies in Chapter 7, Using Add-ons. Writing tests for customizations We will begin by making a simple change: setting the browser window page title and site description. These values are managed as properties called title and description on the Plone site root. We can view these in the Site control panel under Site Setup in Plone. [ 96 ]

As good software developers, we will write automated tests before implementing the functionality. In this case, we will write integration tests that inspect the state of the Plone site after our package has been configured and activated. Chapter 5 By convention, tests go into a module or subpackage called tests. We already have such a module (tests.py) containing some example code, which we will replace with our own test code. Test fixture setup code conventionally lives in a module called testing (testing.py), which may be imported by other code that wishes to reuse a package's test setup. The boilerplate in the tests.py as generated by ZopeSkel at the time of writing uses the Products.PloneTestCase testing framework. Our code will use the newer plone.app.testing framework, which aims to replace Products.PloneTestCase. See the plone.app.testing documentation (http://pypi.python.org/pypi/plone.app. testing) for more details, including a comparison between the two. Before we write the tests themselves, we will add a test layer that configures a shared test fixture for our integration tests. In testing.py, we have: A test layer allows multiple tests to share the same fixture, alleviating the need for each test to set up and tear down a complex fixture. Test layers can also control the lifecycle of individual tests, for example to isolate each test in its own transaction. from plone.app.testing import PloneSandboxLayer from plone.app.testing import applyprofile from plone.app.testing import PLONE_FIXTURE from plone.app.testing import IntegrationTesting from zope.configuration import xmlconfig class OptiluxPolicy(PloneSandboxLayer): defaultbases = (PLONE_FIXTURE,) def setupzope(self, app, configurationcontext): # Load ZCML import optilux.policy xmlconfig.file('configure.zcml', optilux.policy, context=configurationcontext ) def setupplonesite(self, portal): [ 97 ]

Developing a Site Strategy applyprofile(portal, 'optilux.policy:default') OPTILUX_POLICY_FIXTURE = OptiluxPolicy() OPTILUX_POLICY_INTEGRATION_TESTING = IntegrationTesting( bases=(optilux_policy_fixture,), name="optilux:integration" ) This layer uses the PloneSandboxLayer helper from plone.app.testing. It first loads the package's configuration, and then installs its GenericSetup profile. All tests that use the OPTILUX_POLICY_INTEGRATION_TESTING layer will thus be able to assume the package has been configured and its profile applies to the Plone site that is set up by the PLONE_FIXTURE base layer. The tests, in tests.py, look like this: import unittest2 as unittest from optilux.policy.testing import OPTILUX_POLICY_INTEGRATION_TESTING class TestSetup(unittest.TestCase): layer = OPTILUX_POLICY_INTEGRATION_TESTING def test_portal_title(self): portal = self.layer['portal'] self.assertequal( "Optilux Cinemas", portal.getproperty('title') ) def test_portal_description(self): portal = self.layer['portal'] self.assertequal( "Welcome to Optilux Cinemas", portal.getproperty('description') ) If you have not done so already, you should read the documentation at http://pypi.python.org/pypi/plone.testing and http:// pypi.python.org/pypi/plone.app.testing to familiarize yourself with testing concepts and APIs. We should now be able to run the tests. Both of the preceding tests should fail, since we have not yet written the functionality to make them pass. $ bin/test -s optilux.policy Running optilux.policy.testing.optilux:integration tests: [ 98 ]

Chapter 5... Failure in test test_portal_description (optilux.policy.tests.testsetup)... AssertionError: 'Welcome to Optilux Cinemas'!= ''... Failure in test test_portal_title (optilux.policy.tests.testsetup)... AssertionError: 'Optilux Cinemas'!= u'plone site' Ran 2 tests with 2 failures and 0 errors in 0.010 seconds. The actual output is a little more verbose, but these lines tell us that both our tests failed, as expected. Making a change with the extension profile To implement the desired functionality, we create a file inside profiles/default called properties.xml, containing the following code: <?xml version="1.0"?> <site> <property name="title">optilux Cinemas</property> <property name="description">welcome to Optilux Cinemas</property> </site> This was taken from the corresponding file in Products/CMFPlone/ profiles/default, reduced to only the properties that we wanted to change. Plone's base profile is a good place to look for examples of GenericSetup syntax. See also http://plone.org/documentation/ manual/developer-manual/generic-setup. One of the import steps installed with Plone, Site Properties, knows how to read this file and set properties on the site root accordingly. Our tests should now pass: $ bin/test -s optilux.policy Running optilux.policy.testing.optilux:integration tests:... Ran 2 tests with 0 failures and 0 errors in 0.008 seconds. [ 99 ]

Developing a Site Strategy Activating the package Finally, we should verify that we can activate the package through the Plone interface. After starting Zope, we can go to our existing Plone site, log in as a user with Manager rights. Under Site Setup, in the Add-ons control panel, the new package should show up as shown in the following screenshot: We can now activate it, and verify that this causes the title in the browser to change. If we are setting up a new site, we can activate the profile by selecting Optilux Site Policy on the Create a Plone Site screen. Rebuildout, restart, reapply? New Plone developers are often confused about when to rerun buildout, restart Zope, or reapply a profile in Plone. In this chapter, we have made three types of changes that required different types of reinitialization: [ 100 ]

Chapter 5 Changes to the buildout configuration and distribution metadata (in setup. py). These types of changes only take effect once we rerun buildout. Changes to Python and ZCML code. These generally require a Zope restart, although we can often bypass this using plone.reload, as described in Chapter 3, The Development Environment. Changes to a GenericSetup extension profile. These require that the profile be reapplied through the portal_setup tool (or deactivated and re-activated through the Add-ons control panel). They do not require a Zope restart or reload. The exception is the metadata.xml file, which is read at Zope startup only. Summary In this chapter, we have seen: How to create a "policy package" to encapsulate a specific policy for a site How to use a GenericSetup extension profile to customize various aspects of Plone When to rerun buildout, restart Zope, and reinstall a package in Plone As we build new functionality throughout the book, we will continue to add to the policy package. In the next chapter, we will extend it with custom workflow and security settings. [ 101 ]

Where to buy this book You can buy Professional Plone 4 Development from the Packt Publishing website: http://www.packtpub.com/professional-plone -4-development/book. Free shipping to the US, UK, Europe and selected Asian countries. For more information, please read our shipping policy. Alternatively, you can buy the book from Amazon, BN.com, Computer Manuals and most internet book retailers. P U B L I S H I N G community experience distilled www.packtpub.com