Creating an email newsletter using SimpleNews and phplist

Creating an email newsletter using SimpleNews and phplist Maurice Tomkinson (maurice@hopestreetcentre.co.uk) Versions: Drupal Core: 6.22 Simplenews: 6.x-2.x-dev PHPlist: 6.x-1.x-dev I ve been working on adding mailing list functionality to my web site, and it s taken me several weeks to put together a setup that I m happy with. I found quite a lot of help out there on the forums and blogs, but it comes in small chunks, so I thought it might be helpful to others who trying to do something similar if all the steps were drawn together in one place. This is only one of many possible solutions it worked for me, whether it is good for you will depend on what you are setting out to achieve. I wanted to keep my mailing list separate from my Drupal users list, which determined my choice of packages in the end. I started with Simplenews, and liked the way it allowed me to select a list of content nodes from my site, add a table of contents, sort it into a logical order, and assemble it into a newsletter. Starting with existing content, the time taken to assemble an acceptable newsletter is a matter of minutes. What I wasn t happy with was that Simplenews used Drupal s user membership mechanism to manage its mailing lists. Fine for a small group of colleagues, but I m aiming for thousands of subscribers, and I don t want them all logged into Drupal, where a slight error on the permissions table could give them all editing capabilities on my site! I therefore looked into external mailing list engines and came across the PHPlist application, which has an integration module for Drupal (also called PHPlist). Unfortunately as far as I can tell this integration module lacks the content selection capabilities of Simplenews. It has a mechanism for passing the URL of a node to PHPlist, but due to some problem (presumably with my hosting platform) I couldn t get PHPlist s URL placeholder to work. I don t think this is anything to do with Drupal, other PHPlist users who use the package independently of Drupal are reporting similar problems, but I spent many hours trying to find a solution without success. Before giving up on PHPlist I went down the track of trying to use RSS the application has a mechanism for importing an RSS feed and turning it into a newsletter, which would have been great if I could get it to work. Unfortunately I had problems with RSS too PHPlist would accept some feeds but display unhelpful error messages with others. Because the support for PHPlist is not great there isn t the sheer volume of problem reports and fixed that Drupal has I eventually gave up on RSS too. I then realised that I didn t have to use either of these mechanisms. A small tweak to PHPlistbackend.module would allow me to send the full HTML of my content to PHPlist rather than a URL placeholder. I achieved this by modifying the function PHPlistbackend_nodeapi as follows: 'message' => '[URL:'. $url. ']', becomes 'message' => $node->content['body']['#value'],

and $message['message'] = '[URL:'.$url.']'; becomes $message['message'] = $node->body; This allows the full content of my newsletter node to be fed into PHPlist s message database. To make this happen I rely on the PHPlist integration module s features. I won t discuss setting up the PHPlist module as its configuration pages under Site Configuration > PHPlist are self-explanatory, but it s worth noting that the switch to enable its functionality on a specific content type is hidden in the Workflow Settings drop-down (which took a while to find in the documentation). I m actually using a content type called Newsletter issue which was created for me when I installed Simplenews, and has a machine-readable type of simplenews. Thus both modules access the one newsletter content type. I have noticed that there is a warning in the documentation that there may be a compatibility problem between Simplenews and the PHPlist module, and the advice is not to use the two modules together. Perhaps it s because I only use limited functionality of both modules, but I haven t come across this problem yet. Enabling the workflow setting Send as newsletter for the Newsletter issue content type then exposes a set of Newsletter options on the editing page for the Newsletter issue node. These are Newsletter status, Senddate and Lists. The lists selection is the only one I use, and it is populated from the mailing lists created in the PHPlist application itself. One of these has to be selected or the newsletter can t be saved. As soon as Save is clicked on a newsletter issue the PHPlist integration module gets to work and puts the node content into the PHPlist database. It is now ready for sending by the PHPlist application. So by this point I had a mechanism for going from content nodes to newsletter via Simplenews, and newsletter to mailshot via the PHPlist application and its Drupal integration module. So far so good. Unfortunately the newsletter didn t look the way I wanted it and the images in my content nodes were missing. It was time for some theming. I m using the theme blogbuzz, and I therefore have a directory sites/all/themes/blogbuzz which contains the theming information. In there is a file called template.php, and it is this file I modified to do all my customisation. I m not sure if that s the right place, but it worked for me! The starting point for customising the Simplenews newsletter format is in the sites/all/modules/simplenews_content_selection directory and a file called scs.theme.inc. In there is a function called theme_scs_node_output which looks like this: /** * Each selected node goes true this function to create a nice body */ function theme_scs_node_output($node) $output = ''; $output = '<div id="node_'. $node->nid. '">'; $output.= '<h1>'. $node->title. '</h1>'; $output.= '<p>'. node_teaser($node->body). '</p>'; $output.= '<p>'. l(t('read more'), 'node/'. $node->nid). '</p>';

return $output; Following instructions I pasted this into my above-mentioned template.php and renamed it blogbuzz_ scs_node_output. If you are using a different theme you will replace blogbuzz with your theme s name. Later I also ended up overriding a second theme function theme_scs_newsletter_output which I copied and renamed in a similar way. To get Drupal to recognise these new functions the Theme Cache needs to be refreshed (easy to forget!) Eventually I realised that once Drupal has recognised the new functions it s not necessary to keep refreshing the cache every time you change the PHP code. Getting the images to appear was much harder than I expected. Drupal has more than one mechanism for displaying images, and if you re looking at code snippets it s easy to get confused about which one is which. A lot of the snippets relate to CCK ImageField objects which are inserted into content types using the Manage Fields mechanism. I don t do it this way, I use Image Attach, and the problem is that the two mechanisms are coded differently, so that a piece of code that works for one won t work for the other. The images created by the Image Attach mechanism are stored as separate nodes, and the content node just stores a nid for the image node. To get the image node ($image) from the content node ($node) I use the code: $image = node_load($node->iids[0]); Another thing to remember about Image Attach is that it doesn t store one picture, but many of different sizes. This is controlled by the settings page Site Configuration > Images > Files and Sizes. By default there are three sizes, Original, Thumbnail and Preview, but none of these were the size I wanted so I created another size called Newsletter which I scaled to 250x250. The relative URL for this new image size can be obtained by this line of code: $imageurl = $image->images['newsletter']; One small quirk here I found that although the name of the size newsletter has an initial capital in the Files and Sizes page it needs a lower case string to access in the code. Another thing to remember is that once your content leaves Drupal and flies off into the aether any relative URLS will be broken, so $imageurl needs to be converted to an absolute URL so that your email can retrieve it from your site. Drupal provides a built-in function url() to do this: $absimageurl = url($imageurl, array('absolute'=>true)); In passing, I should mention that printing out the node contents in a comment helped a lot while I was working out what was going on. The following code snippet does this ($output is the string that stores the newsletter content as it s being created, and you can replace $node with any object you like): $output.= '<!--' ; // treat the output as a comment $output.= print_r ($node, TRUE); // output the contents of $node $output.= '-->'; // close the comment block

The comment block can then be viewed by switching to Source mode (if you have it) in your WYSIWYG editor. When Simplenews has created a newsletter it stores it in a newsletter node and opens up an editing page so that you can make any adjustments you want. Unfortunately I discovered that the HTML that was appearing in my editor was different to the HTML that my PHP code was emitting. Unwanted tags were appearing all over the place! I use CKEditor as my WYSIWYG plugin, and although I can t be totally sure it appears that CKEditor was taking it upon itself to modify my HTML. This mainly affected font styles and anchor tags. What should have been: <DIV><H1>My Newsletter</H1></DIV> was coming out as <DIV><H1><SPAN style="font-family: serif; FONT-SIZE: 12px">My Newsletter</SPAN></H1></DIV> The additional SPAN tag was overriding the font size that I wanted, and there seemed to be nothing I could do about it! Eventually I discovered that it was better to remove all the CSS formatting (eg. the <H1> tags) and substitute inline formatting, because the CSS can get ignored by email clients. Once I removed the CSS and inserted my own style instructions, CKEditor seemed to settle down and leave my code alone. Another problem with CKEditor was with anchor tags. Once of the nice features of Simplenews is that it provides an optional contents list, with local links to anchors in the body of the newsletter. The way it does this is to emit a <DIV> tag with an id at the start of each component node, and sets up a jump using an <A> tag from the contents list: <A href="#node_192">contents list link to node 192</A> should jump to <DIV id= node_192 Unfortunately, although this is technically correct, CKEditor seems not to like the id in a DIV statement, and responds by expanding href= #node_192 to href= http://example.com/node/401/edit#mode_192. If an email reader clicked on this link it would take them back to editing the newsletter! (Hopefully they wouldn t have the permission to do so, but it s very messy!) After much trial and error I found that replacing the id in the DIV tag with an alternative HTML form: <A name= node_192 > kept CKEditor happy and stopped it trying to convert my href to an absolute URL. As one final touch I created a pair of unpublished nodes on my site to contain a header and a footer for the email (so much eaiser to manage in Drupal than in the PHPlist application).

To summarise, the complete listing of customised code I produced is here: // Helper function to get node object from its aliased url function get_node_from_alias($pathalias) $pathsource = drupal_lookup_path('source', $pathalias); $temp = explode("/", $pathsource); $nid = $temp[1]; $node = node_load(array('nid' => $nid)); return $node; // blogbuzz_scs_node_output formats an individual node function blogbuzz_scs_node_output($node) $output = ''; $imagetag = ''; $image = node_load($node->iids[0]); $imageurl = $image->images['newsletter']; // Retrieve the url for the preview version $temp = 'node/'.$node->nid; $absurl = url($temp, array('absolute'=>true)); if (!empty($imageurl)) $imagetag = '<img src="'. url($imageurl, array('absolute'=>true)). '" />'; $output = '<div style="background-color:#fff5d2;">'; $output.= '<a name="node_'.$node->nid.'"></a>'; $output.= '<div style="background-color:#ccbbbb; padding-top: 5px; paddingbottom: 5px;">'; $output.= '<SPAN style="font-size: 20px; FONT-FAMILY: Georgia, \'Times New Roman\', Times, serif; color: #660000">'; $output.= l($node->title, $absurl); $output.= '</span></div>'; if (!empty($imageurl)) $output.= '<div style="float: right; margin-left: 1em;">'; $output.= l($imagetag, 'node/'.$node->nid, array('html'=>true)); $output.= node_teaser($node->body); $output.= '<br>' ; $output.= l(t('read more...'), $absurl); $output.= ' <div style="clear: both;"></div>'; return $output; // blogbuzz_scs_newsletter_output formats a complete newsletter. function blogbuzz_scs_newsletter_output($nodes, $toc) $body = ''; $contents = ''; $nodeheader = get_node_from_alias('newsletter/header'); $nodefooter = get_node_from_alias('newsletter/footer'); if ($nodeheader) $output.= $nodeheader->body; $output.= '<div style="background-color:#ccbbbb; padding-top: 5px; paddingbottom: 5px;">'; $output.= '<SPAN style="font-size: 24px; FONT-FAMILY: Georgia, \'Times New Roman\', Times, serif; color: #660000">'; $output.= 'My Newsletter<br>'; $output.= '</span></div>'; $titles = array(); foreach ($nodes as $node) // Node information $body.= theme('scs_node_output', $node);

if ($toc) // ToC (if required) $output.= '<p>table of Contents</p><div>'; foreach ($nodes as $node) // Node information $contents.= '<a href="#node_'. $node->nid. '">'. $node->title. '</a><br>'; $output.= $contents; $output.= $body; if ($nodeheader) $output.= $nodefooter->body; return $output;