Using OpenEMM as a frontend to traditional mailing lists Jim Dutton 03/2013, 04/12/2013, 04/29/2013, 06/07/2013

Transcription

1 The intended, or traditional, use of OpenEMM is as a closed system providing T+C +(SS)U functionality. It is also possible to use OpenEMM as a T+C system that can act as a frontend to traditional/existing (sendmail) simple/complex mailing lists, Listserv mailing lists, or even Usenet/Network/Google (news)groups that allow posting by . T: templates C: content SS: self-serve U: users/recipients Obviously, for regular, plain text, non-graphical s to traditional mailing lists, OpenEMM is not necessary. However, for those situations where a graphical is intended or desired, and is allowed, for transport through traditional mailing list systems, then OpenEMM's T+C functionality can be very useful. Note that this usage is not intended to provide recipient profile self-control or responses through OpenEMM. All user control is under the framework of the external mailing system. Just as would be done with regular/traditional OpenEMM use, one creates templates and their corresponding content modules. No target groups are created, however (though that may be possible). Simple mailing lists are created and named after the external mailing list/newsgroup. The only recipients created are pseudo-users whose definition will look something like: FirstName: <mailing list/newsgroup name>< -test -admin > (where the two qualifiers might be used for only for those specific mailing types) LastName: MailingList or Newsgroup < address for mailing list/newsgroup> Mailing List: enabled for OpenEMM mailing list that matches FirstName value (without -test or -admin ) The -test/-admin suffixes can be (optionally) used to create more specific purpose identified recipients, not just what is selected for their mailing options. Using these would then result in two or three recipients for a particular mailing list allowing one to have a test recipient, and admin recipient and a normal recipient. Another way to achieve the same affect is to change the one (and only) recipient and their mailing options, but then one has to go back and forth to make these changes to perform different types of mailings (test, admin, normal). Page 1 of 14

2 The Mailing definition should be something like: Subject: <appropriate for the particular mailing> Sender <anything reasonable; perhaps related to the list or group> Sender full name: <name of mailing list/newsgroup> Reply-to & Reply-to full name: <anything reasonable, or nothing> It is expected that these traditional delivery systems are on another host than the one where OpenEMM is installed. For complex sendmail mailing lists (double alias substitution with :include: ), this is a requirement as the mailing process associated with OpenEMM (as of 03/2013) is unable to process such e- mail addressing schemes. Obviously, sendmail mailing lists require the full Sendmail software to be installed (which is not always available in base Linux installations, but can be easily added: sendmail plus sendmail.cf ). Once the OpenEMM recipient, mailing list, and mailing are created, the generated is then transported to and through the traditional mail/newsgroup systems. This results in a one-to-many fan-out outside of OpenEMM. Page 2 of 14

3 Sample External/Sendmail Setup (on a separate host) In the sendmail aliases file: <list group name>: :include:/etc/mail/mailing_lists/<list group name>.list Note the the :include: tag/parameter must be entered as shown above space before but no space after. In the sendmail /etc/mail directory: mailing_lists/<list group name>.list In the list file: <list group remote address #1> <list group remote address #2>.. <list group remote address #N> General file/owner (user,group) permissions: aliases: 0644; root, root /etc/mail: 0755; root, root /etc/mail/mailing_lists: 0755; root, mail list file: 0644; root, mail Red Hat, and other Linux systems, define /etc/passwd/mail as userid:groupid 8:12, with /etc/group/mail:12, and sendmail.cf/defaultuserid as 8:12. Some mailing functions are carried out as the mail userid though the sendmail daemons will be run as root or smmsp. Other systems do not use this convention so the above uses of the mail group need to be tailored for the specific system being used. The gist of all of this is that the mailing_lists directory and the mailing list files need to be readable by the mail (or equivalent) userid. A simple way to verify what userid is being used to access the mailing list files is to shutdown the sendmail receiver (or all sendmail daemons) and run sendmail in debug mode, as shown below, and then try to send an OpenEMM mailing pointing to one of the above :include: mailing lists and see what happens. Directories and files above may be changed as necessary. Page 3 of 14

4 Note that some of the debug options used here may not be necessary or even useful, but this is a list that I used and its output solved my problem of trying to figure out who was trying to access the mailing list files: sendmail -bd -D /tmp/sendmail.debug -d6.1,8,1,11.1,13.1,20.1,27.1,27.5 Then as the smtp host was processing the double-aliased mail sent out by OpenEMM, I found the following error messages (in the volume of debug messages): include(/etc/mail/mailing_lists/<filename>.list) ruid=0 euid=0 include: not safe (uid=8): Permission denied So, userid 8 could not access the mailing list file. That in turn (for Linux) turned out to be the mail userid which also has a mail groupid. This then led to the file/owner permissions settings above. Again for OTHER systems, you may not see anything relating to a mail userid. If there is a file/permission access problem, then the above error message may identify who the mailing process was attempting to act as, and whose group name needs to be used similar to above. Linux Sendmail Administration page on DefaultUser The bottom line is that the mailing list/newsgroup list files need to be readable by the mailing process - whatever that is. Technically, the sendmail :include: file can reside anywhere on the host, but it is still restricted to the mailing process read access requirement. I chose to put it in /etc/mail as that is where most things sendmail are kept. Last but not least, the originating OpenEMM server/host will need to be defined as a relay host on the transmitting (relaying) SMTP server/host in the /etc/mail/relay-domains file. Page 4 of 14

5 Benefits to an OpenEMM frontend 1. Template developers and their templates can be protected from content developers 2. Content developers and their content creation can't change (accidentally or incidentally) any templates 3. Content developers may not have to worry about or develop templates 4. The OpenEMM C.M.S method provides a simple yet sufficient editing environment for most simple to moderately complex HTML documents, or one can easily paste in content developed in an external editor 5. OpenEMM provides a choice of fixed layout templates or dynamically arranged template layouts 6. OpenEMM provides a central storage facility for templates so that individuals don't have to manage them on their own, and many (authorized) individuals can make use of the same templates 7. Stored templates (at least static templates) can be easily used as the basis for new templates, again stored in a convenient central location Think of OpenEMM as a type of C.M.S. (content management system). One could use OpenEMM strictly as a frontend to external mailing delivery systems, or use OpenEMM for its full design purposes and still include the ability to frontend external mailing systems. Side note: one of the benefits of the Sendmail :include: mechanism is that with the proper ownership and permission settings, the referenced file can be placed just about anywhere, and can be changed at any time without having to recycle Sendmail (though it would be best not to make any changes during a ongoing mail delivery cycle using that file). This can provide for dynamic and possibly large scale, en masse, recipient information maintenance. Where there are large clientele changes daily (which are not predicated by user choice but by organizational human resource departmental data flow) and all mailing list management is taken care of outside of OpenEMM then where graphical to external mailing list systems is allowed and desirable, they can benefit from the use of OpenEMM. Page 5 of 14

6 Providing sendmail comment/full name strings for outbound mail Added: 06/07/2013 Most mail delivered through traditional mailing lists (ML) will have the name of the list either in the Subject header or the To (recipient) header. In most cases, this is just fine, and is the norm. Sometimes, however, it may be nice, or desired, to add a sendmail comment string to the recipient address such that it now consists of two parts: an (RFC822) electronic mail address and an (RFC822) electronic mail comment (full name) string. According to RFC822 (O'Reilly sendmail book, Basic Textual Canonicalization), this combination can be formatted in one of two ways: comment/full name string < address> address (comment/full name string) When the full name/comment string is included, it usually is presented, or seen, first in the recipient header record and in the mail client recipient field/column. For general mailing list mail, this is not normally used, but in our situation where the mailing lists are not user-subscriptions but enterprise-assigned (ie; they are outbound only distribution lists), it is nice to offer a descriptive comment to better identify the target audience. Also, the actual mailing list name may be an abbreviated form of something that may not be meaningful by itself (at least, not to humans). For example, we are an academic institution with multiple client groups as well as multiple campuses. Some sample mailing lists might be (pseudo mailing list name, pseudo mailing list description): oiuwer <campus A> Employees oiuers <campus B> Faculty oiuewrsf <campus A, campus B, campus C> Students Since these are outbound delivery vehicles only, in our case, having the ML description used as a full name value provides a better group identification which will be more meaningful than the normal ML name, and should help to reduce spam mail throwaway as the target recipients should recognize their general group description as given by the ML description. For sites implementing OpenEMM + remote SMTP server for delivery to traditional mailing lists where they are user-subscription based, this process may not be of much interest nor of much added value. For organizations which use traditional mailing lists for similar purposes as we do, regardless of the type of organization, the remainder of this discussion may be of great interest. The trick occurs in the remote SMTP server. Page 6 of 14

7 As a quick review, the OpenEMM->external mailing list, in this document, creates a single mail file forwarded to a remote (external) SMTP server which contains the actual mailing list configuration (using sendmail). Using sendmail, its external filter hook ( milter ), and a simple Perl program, it is a trivial matter to inject an e- mail full name/comment string into the single OpenEMM mail To: header before the mail gets distributed. Prerequisites: sendmail-8.12 (including sendmail.cf); best: v8.14 (or newer) sendmail-milter (external filter hook API interface) perl-sendmail-milter (Perl interface that communicates with the milter hook ) mailing lists defined in /etc/aliases (simple entry or complex include) file containing mailing list names and descriptions file containing local user(id)s whose mail should be ignored (eg; root) sendmail semodule for selinux installations References: O'Reilly sendmail book, Chapter 26: X(Milters) Configuration Command sendmail (source) documentation: README.libmilter perl-sendmail-milter documentation: Sendmail::Milter man(ual) page, doc/readme, sample.pl The sample.pl program is very helpful in understanding how this facility works and should be executed first, after installing the first three prerequisites, before dealing with the ML data (and selinux). Note that the testing process may be disruptive to the normal sendmail process. For implementing sites not using selinux, please ignore all selinux related items. Whilst we used a ML key file to provide the descriptions with a particular format, which is reflected in the runtime perl milter script - you should determine your own format specifications for this file and make the necessary adjustments to the sample runtime script provided here. There may be other adjustments in the various configuration items in this process, detailed later, that may be necessary or desirable for an implementing site. One does not have to use exactly the same choices that we made. Some perl-sendmail-milter messages are sent to Syslog by default. Page 7 of 14

8 NOTE: all of this is NOT to be done on the main OpenEMM server it must be done on a separate SMTP server! 1. Install all of the required sendmail components listed above 2. Test the sendmail milter with the supplied sample.pl script that should come with the perl-sendmail-milter package 1. stop the sendmail service 2. adjust sendmail configuration as detailed in the package README file 3. start the perl-sendmail-milter sample.pl script (do not put it in the background) 4. restart the sendmail process 5. test sending mail to the SMTP server (ie; use local mail client, such as alpine) observe the results (note: you may or may not have to deal with selinux issues) 3. Once you see how the milter works (look at the milter output AND the system mail log file), prepare to implement the AddRecipientComments_milter.pl script 1. Create or modify mailing lists and define in /etc/aliases (or wherever the system sendmail aliases file is kept) 2. Create the key (ML description file) in /etc/mail somewhere (don't make descriptions really long like mini-paragraphs ) as root 3. Adjust the runtime perl milter script, AddRecipientComments_milter.pl, as needed with regards to the key file above; place script in /etc/mail 4. Make adjustments to the sendmail configuration using the sendmail.cf or sendmail.mc file 5. Apply the selinux policies if using selinux (two steps: semanage and semodule) 6. Create /etc/mail/local_users file with all system/local user(id)s: one line with a blank AFTER each userid Page 8 of 14

9 7. Restart the sendmail process; resolve any problems that occur 8. Test mail from OpenEMM and observe SMTP server mail log file as well as delivered mail (use a, small, test mailing list for this as you may wish to send out test mails multiple times) 4. If everything (mail, system, selinux, delivered mail, whatever) is working and looking OK, then it is time to modify whatever system sendmail startup script is used to include the perl-sendmail-milter runtime script 1. Back up the existing system sendmail runtime script(s) 2. Using the following code examples, modify the sendmail runtime script(s) accordingly 3. NOTE: the perl-sendmail-milter runtime script must be started before sendmail, and there needs to be a few seconds delay before starting sendmail as this script creates a socket file which has to exist before the milter connection in sendmail will operate properly! 4. Only relevant parts of the existing sendmail startup script will be presented here 5. Note that the daemon command (in Linux) wasn't used as it required other parameters that aren't needed/don't work for this script start() { ret=0 # Start perl sendmail milter - has to open socket before sendmail daemons run if [! -f /var/run/sm-milter.pid ]; then echo -n $"Starting sm-milter: " /usr/bin/perl /etc/mail/addrecipientcomments_milter.pl AddRecipCommFilter /etc/mail/sendmail.cf& RETVAL=$? echo let ret+=$retval /bin/sleep 3 # need time for socket file to be created before starting sendmail else milter fails fi # Start daemons. # ret=0 updateconf echo -n $"Starting $prog: " daemon /usr/sbin/sendmail $([ "x$daemon" = xyes ] && echo -bd) \ $([ -n "$QUEUE" ] && echo -q$queue) $SENDMAIL_OPTARG RETVAL=$? echo [ $RETVAL -eq 0 ] && touch /var/lock/subsys/sendmail let ret+=$retval... Page 9 of 14

10 stop() { # Stop daemons. if [ -f /var/run/sm-client.pid ]; then echo -n $"Shutting down sm-client: " killproc sm-client RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f /var/run/sm-client.pid [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/sm-client fi echo -n $"Shutting down $prog: " killproc sendmail RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/sendmail # Stop sendmail perl milter if [ -f /var/run/sm-milter.pid ]; then echo -n $"Shutting down sm-milter: " killproc sm-milter RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f /var/run/sm-milter.pid && /bin/rm -f /var/run/f1.sock fi return $RETVAL 1. Note any changes needed in the runtime script line which parses the ML description file (yes you need to understand regex ): ($MLlistname,$MLdescription) = $_ =~ /([a-za-z0-9_-]+)\s+name:\s*(.*)/; 2. Make necessary changes to the perl-sendmail-milter runtime script 3. Restart the sendmail services 4. Test OpenEMM mail (log files and delivered mail) The perl-sendmail-milter runtime script that we used is included on the next pages. Page 10 of 14

11 #!/usr/bin/perl -w # AddRecipientComments_milter.pl - insert sendmail "full name" into To header # Copyright (C) 2013 Jim Dutton # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program. If not, see < use Sendmail::Milter; use Socket; use Sys::Syslog; sub my_envrcpt_callback { my $ctx=shift; my $RecipientAddress=shift; $ctx->setpriv(\$recipientaddress); return SMFIS_CONTINUE; sub my_eom_callback { # get mailing list descriptions (mail_lists.key) # change regex expression below to match your "key" file format! open(mlkey,'</etc/mail/mail_lists/key') die "500 cannot open /etc/mail/mail_lists/key"; my %MLhash=(); my $MLlistname; my $MLdescription; while (<MLkey>){ chop; ($MLlistname,$MLdescription) = $_ =~ /([a-za-z0-9_-]+)\s+name:\s*(.*)/; $MLhash{$MLlistname='"'. $MLdescription. '"'; close(mlkey); Page 11 of 14

12 # get list of local users whose mail should be ignored; "local_hosts" is a non-sendmail file # there should always be "root" in this file as that is used by many system processes # file consists of one line, each user(id) FOLLOWED by a blank (eg; blank at end of file) open(localusers,'</etc/mail/local_users') die "500 cannot open /etc/mail/local_users"; $LOCALusers=<LOCALusers>; close(localusers); my $ctx = shift; my $RecipientAddress=$ctx->getpriv(); my ($MLlistname2,$MLdomain); ($MLlistname2,$MLdomain) = ${$RecipientAddress =~ /<(.*)@(.*)>/; # if local part of address is a "local user", simply return (eg; do nothing) if (index($localusers,$mllistname2) > 0){return SMFIS_CONTINUE; # if the mail addressee is a known mailing list name, change the To header # otherwise, don't make any change and just return if (exists $MLhash{$MLlistname2){ $ctx->chgheader("to",0,$mlhash{$mllistname2. " ". ${$RecipientAddress); else { syslog("info","%s","mllistname ($MLlistname2) not found in MLhash - no header changed"); $ctx->setpriv(undef); return SMFIS_CONTINUE; my %my_milter_callbacks = ( 'envrcpt' => \&my_envrcpt_callback, 'eom' => \&my_eom_callback ); BEGIN: { openlog("addrecipcommfilter","nofatal","mail"); open(milterpid,'>/var/run/sm-milter.pid') die "500 cannot open /var/run/smmilter.pid"; print MilterPID $$ ; close(milterpid); Sendmail::Milter::setconn("local:/var/run/f1.sock"); Sendmail::Milter::register("AddRecipCommFilter",\%my_milter_callbacks, SMFI_CURR_ACTS); syslog("info","%s","starting Sendmail::Milter $Sendmail::Milter::VERSION engine."); Sendmail::Milter::main(); Now all mailings listed in the description file should have a recipient ( To: ) header that begins with the ML description text. This is all that this process really does: insert the description text as a sendmail full name. The description text may be in Mixed Case (which will be retained). Page 12 of 14

13 For those trying to implement this in an enforcing mode selinux system, the following information may be necessary: Change selinux file context for runtime script semanage fcontext -a -t etc_mail_t /etc/mail/addrecipientcomments_milter.pl chcon -t etc_mail_t /etc/mail/addrecipientcomments_milter.pl Define selinux policies using sendmail.te source file: module sendmail 1.0; require { type sendmail_t; type var_run_t; type initrc_t; class sock_file { write getattr ; class unix_stream_socket connectto; #============= sendmail_t ============== #!!!! This avc is allowed in the current policy allow sendmail_t initrc_t:unix_stream_socket connectto; #!!!! This avc is allowed in the current policy allow sendmail_t var_run_t:sock_file getattr; allow sendmail_t var_run_t:sock_file write; checkmodule -M -m -o sendmail.mod sendmail.te semodule_package -m sendmail.mod -o sendmail.pp semodule -i sendmail.pp The te (type enforcement) file was generated by the audit2allow script. Verify the results after applying this module and running the perl-sendmail-milter runtime script. Make adjustments as needed. Note: instead of chcon above, one could also use restorecon. Page 13 of 14

14 Closing Comments 03/2013 We chose to use this method of OpenEMM being an formatting and repository tool (front-end) that sent a single deliverable to traditional e- mail mailing lists because we have a very large population turnover which can happen on a daily basis. This population isn't coming to us to request information (ie; self-serve signup for mailings) they are already known from other enterprise processes and are automatically targeted (for certain enterprise mailings). Also, we already had established multiple mailing lists and we didn't have to do anything to/with OpenEMM to be able to continue to use them, which also meant that we won't have to maintain a large database of recipients. At the same time, using OpenEMM in the T + C mode provides us with the opportunity to take any number of fixed template structures and allow one or more designated individuals (who may not be well versed in HTML or CSS) easy methods to enter content without having to know or worry about content layout. We may exploit the OpenEMM dynamic template features in the future, but for right now we really need the fixed template features. Caveats One more time specifics above may (need to) be different for your system, so take all of the above with a grain of salt. SeLinux systems may run into selinux ACL requirements that have not been discussed here. Keep this is mind, however, as this may cause some permission denied errors where everything else is correct. If this is the case, don't turn off selinux learn how to use it (eg; chcon, semanage, setroubleshoot, policycoreutils, auditd, audit2allow, semodule, et cetera). Page 14 of 14