Advanced Settings Oct 10, 2011 Igor Seletskiy CEO, CloudLinux
Linux OS based on RHEL source RPMs Binary compatible with RHEL 5.x/6.x and CentOS 5.x/6.x Made for Shared Hosting Companies Focus on Stability and Security Excellent, free 24/7 support Affordable for Companies of any Size CloudLinux delivered patches for several local exploits days before RHEL and CentOS
Lightweight resource limits CPU/Concurrent Connection/Memory limits Transparent to administrator Easy to deploy to CentOS/RHEL servers No need to setup per customer limits Easy to monitor resources usage on per user bases
One customer usually the cause of majority of downtimes Limiting resource usage for a customer his usage spikes will prevent issues for other customers Hard & expensive to track Takes time to track, which results in downtime for the server.
Better stability No server slow downs No need to suspend customers due to resource abuse Simplifies up sell to higher plans / VPS Ability to track usage on per customer bases Less support Improved security Better density
We deployed CloudLinux on our shared hosting servers, and we have seen immediate improvement in reliability. Ditlev Bredah, CEO UK2 Group We deployed CloudLinux on our shared hosting servers, and we have seen immediate improvement in reliability. Now we want to offer the same solution to our dedicated server customers. Emmanuel Vivar, CEO HostDime
Resource Limits CPU Memory Connections IO Priorities (limits coming soon) Ability to change limits on the fly Ability to see current & historic usage Easy way to pin-point abusers
Kernel level technology modified CPU scheduler cgroups/bean counters Apache module talks to kernel Works with any technology: cgi, mod_php, fcgid, MPT ITK, ruid2, LiteSpeed, etc PAM module for shell/cron jobs
/etc/container/ve.cfg lvectl ubc /proc/lve/list /proc/user_beancounters /etc/sysconfig/lve
CloudLinux 5.x Based on 2.6.18 kernel UBC memory limits disabled by default Migration thread per core Live ncpu change since lve0.8.42 kernel gresecurity TPE patch CloudLinux 6.x Based on 2.6.32 cgroups memory limits always enabled No migration processes Live ncpu changes No grsecurity TPE patch (yet) Performance optimization still in progress
/etc/sysctl.conf fs.proc_can_see_other_uid=1 fs.proc_user_gid=0
Different way to calculate 1 per LVE, instead of per process # sysctl -w kernel.full_loadavg=1 Accounting for uninterruptable processes
0 means disabled mod_php, LiteSpeed not supported Needs to fork to start counting Virtual memory vs real memory usage. What happens when user hits memory limit
Number of cores per LVE 1 migration thread per enabled core Virtualized /proc/cpuinfo
# lvectl set XX ncpu 2 force Only CL6 or lve0.8.42 & later
Smallest limit wins 1 core server, ncpu=1, cpu=25% -- true limit = 25% 2 core server, ncpu=1, cpu=25% -- true limit = 25% 4 core server, ncpu=1, cpu=25% -- true limit = 25% 8 core server, ncpu=1, cpu=25% -- true limit ~ 12% 16 core server, ncpu=1, cpu=25% -- true limit ~6% 16 core server, ncpu=2, cpu=25% -- true limit ~12% 16 core server, ncpu=4, cpu=25% -- true limit = 25%
No more NCPU CPU will automatically adjust NCPU parameter --core specifying speed on per core bases. 50% -- half a core, 150% -- 1.5 cores.
# lveinfo --from='2010-10-10' --to='2010-10-15' -o cpu_max --display-username ID acpu mcpu lcpu aep mep lep MemF MepF web2 5 10 10 14 25 25 4 342 web1 3 8 10 6 18 25 0 0 ID acpu mcpu lcpu aep mep lep MemF MepF User or LVE id Average CPU usage Max CPU usage CPU Limit Average Number of Concurrent Connections Max Number of Concurrent Connections Limit on Concurrent Connections Out Of Memory Faults Concurrent Connections faults
508 Concurrent connections (maxentryprocs) 500 Either server/script error or memory limits Use lve-stats by-fault to figure out the issue CPU limit will slow down the site, but it will not cause it to fail.
physical memory instead of virtual memory Processes share memory Allocated memory doesn t mean used memory Copy on write OOM killer will kill processes if LVE hits memory limit
Number of Processes CPU Weight Stats in /proc/user_beancounters
lvestats-server binary Polls data ever 5 seconds - Updates /var/lve/info Writes to database every minute Sqlite db /var/lve/lveinfo.db Can write to centralized MySQL/PostgreSQL Data is read via lveinfo Centralized portal coming soon
/etc/sysconfig/lvestats db_type: postgresql, mysql, (sqlite) server_id unique id for the server Required DB server Will have to create user/db/tables manually
Part of lve_wrapper package Allows to run anything inside LVE # lve_suwrapper 300 service mysql restart Useful for any system background/cron jobs
Cron Vixie-cron /etc/sysconfig/crond LVE_DISABLE LVE_START_UID Will be supplanted by pam_lve
session required pam_lve.so 500 /etc/pam.d/sshd /etc/pam.d/crond works on PAM level When you su/sudo to root from regular user you will still be inside LVE
httpd.conf, modhostinglimits.conf AllowedHandlers Support regexp Daemon based software: mod_fcgid, mod_cgid, other APR patch to start new processes via suexec
Tries to use all the cores on the server fix by limiting the number of threads to the available processors to 1. This can be done by editing config.xml & policy.xml files /usr/local/lib/imagemagick-[version]/config/policy.xml /usr/local/lib/imagemagick-[version]/config/config.xml Setting: <policy domain="resource" name="thread" value="1"/>
# yum install yum-protectbase Edit /etc/yum/pluginconf.d/rhnplugin.conf Add: [cloudlinux-x86_64-server-5] protect = 1 [cloudlinux-base] protect = 1
You can meet us at booth #4 If you rate my survey, I'll hook you up with $20 cpcache. Go to this address to take the survey: http://go.cpanel.net/b33, and come up to the podium once you've completed it.