Solution for Recording and Reporting Employee Web Activity SGOS 6.5
Third Party Copyright Notices 2014 Blue Coat Systems, Inc. All rights reserved. BLUE COAT, PROXYSG, PACKETSHAPER, CACHEFLOW, INTELLIGENCECENTER, CACHEOS, CACHEPULSE, CROSSBEAM, K9, DRTR, MACH5, PACKETWISE, POLICYCENTER, PROXYAV, PROXYCLIENT, SGOS, WEBPULSE, SOLERA NETWORKS, DEEPSEE, DS APPLIANCE, SEE EVERYTHING. KNOW EVERYTHING., SECURITY EMPOWERS BUSINESS, BLUETOUCH, the Blue Coat shield, K9, and Solera Networks logos and other Blue Coat logos are registered trademarks or trademarks of Blue Coat Systems, Inc. or its affiliates in the U.S. and certain other countries. This list may not be complete, and the absence of a trademark from this list does not mean it is not a trademark of Blue Coat or that Blue Coat has stopped using the trademark. All other trademarks mentioned in this document owned by third parties are the property of their respective owners. This document is for informational purposes only. BLUE COAT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. BLUE COAT PRODUCTS, TECHNICAL SERVICES, AND ANY OTHER TECHNICAL DATA REFERENCED IN THIS DOCUMENT ARE SUBJECT TO U.S. EXPORT CONTROL AND SANCTIONS LAWS, REGULATIONS AND REQUIREMENTS, AND MAY BE SUBJECT TO EXPORT OR IMPORT REGULATIONS IN OTHER COUNTRIES. YOU AGREE TO COMPLY STRICTLY WITH THESE LAWS, REGULATIONS AND REQUIREMENTS, AND ACKNOWLEDGE THAT YOU HAVE THE RESPONSIBILITY TO OBTAIN ANY LICENSES, PERMITS OR OTHER APPROVALS THAT MAY BE REQUIRED IN ORDER TO EXPORT, RE-EXPORT, TRANSFER IN COUNTRY OR IMPORT AFTER DELIVERY TO YOU. Americas: Blue Coat Systems, Inc. 420 N. Mary Ave. Sunnyvale, CA 94085 Rest of the World: Blue Coat Systems International SARL 3a Route des Arsenaux 1700 Fribourg, Switzerland
Contents Solution: Record and Report Employee Web Activity 4 Configure FileZilla FTP Server 4 Enable Access Logging 5 Upload Access Logs to the Reporter Server 6 Create a Reporter Log Source 9 View User Web Activity Reports 12 User Behavior 13 Security 13 Web Application Reports 14 Access Logging Troubleshooting 15 Why is the ProxySG uploading logs so frequently? 15 3
Record and Report Employee Web Activity Solution: Record and Report Employee Web Activity As employees browse the Web, the ProxySG appliance records and stores browse activity data in Access Logs. These logs can be sent to a reporting application, such as Blue Coat Reporter, which provides graphical representations of Web use in your enterprise. Your IT and Human Resource personnel can analyze these reports and adjust Web use, application, and network policies accordingly. This solution provides steps to configure the ProxySG to upload (FTP) two Access Log formats (HTTP/S and streaming) for use with Blue Coat Reporter. This procedure assumes that you have a supported and dedicated Windows or Linux server configured and ready to receive uploaded Access Logs. If you require information about additional or custom formats, consult the Access Logging chapters in the Blue Coat SGOS Administration Guide for your SGOS version. 1. Verify that you have the Reporter location information recorded. Element Value Staging Server Type Windows IP Address Username Password Folder Linux Dedicated, stand-alone server? Yes No (requires FTP software) 2. Configure FileZilla FTP Server Only required if you do not have an FTP access log staging server; for example, you installed Reporter on the same server that receives the logs. 3. Enable Access Logging. 4. Upload Access Logs to the Reporter Server. 5. Create a Reporter Log Source. 6. View User Web Activity Reports. Configure FileZilla FTP Server If you do not have an FTP server that can serve as the staging server for the Access Logs, you can install open source FTP server software on the same server on which Reporter is installed. You can use any similar software, but for demonstration purposes, this section describes the FileZilla FTP server software. 4
1. Download the Filezilla FTP server from http://filezilla-project.org/download.php?type=server. Note: This link is valid as of the date this document was published. The URLs are subject to change without notice. If the link doesn't work, use your preferred search engine to find the FileZilla FTP server. 2. Install the Filezilla FTP server software. Accept the application defaults. 3. Create a directory to stage Access Logs. For this example, the files are staged in the D:\ftp\proxysg\ directory. 4. In the Filezilla server window, click Edit > Users. This displays the current users (none), setup, and configure new users. 5. On the General page (left-side area), click Add under Users. In the pop-up dialog, enter the FTP account name. This example uses proxysg as the account name. Because the group is optional, you are not required to make that user a member of a group. 6. Perform the following. a. In the Account Settings area, verify that Enable Account is enabled. b. Select Password and enter a password for the newly-created proxysg. For security purposes, make the password complex. This example uses bluecoat as the password. 7. Perform the following. a. Click the Shared Folders page. b. Click Add. c. Walk the file system directory tree to D:\ftp\proxysg\ and click OK. d. For files and directories, give that user all file rights (Read, Write, Delete, Append) and all directory rights (Create, Delete, List, + Subdirs) to D:\ftp\proxysg\. Verify that D:\ftp\proxysg\ has a capital H next to it. If not, highlight the directory and click Set as home dir to make that is the home directory for that user. When the proxysg FTP user logs into the FTP server, the root directory for that user is D:\ftp\proxysg\ and that user cannot go any higher in the directory tree. e. Click OK to save the user. Note: The Speed Limits and IP Filter pages are optional and not discussed in this section. You can implement them at your own discretion; however, Blue Coat recommends that you not implement any speed limits or IP filters until after everything else is configured and running correctly. Next Step: Enable Access Logging Enable Access Logging When you enable Access Logging, the ProxySG appliance begins to record all employee-initiated web activity into a series of compressed files. The bcreportermain_v1 Access Log format is for HTTP/S traffic and the bcreporterstreaming_v1 format is for streaming media traffic. These formats contain, among others, the fields that provide user identification, date/time, web content category, and actions taken (such as policy verdict). 1. Log in to the ProxySG Management Console. 2. Verify that the main log defaults to the bcreportermain_v1 format. 5
Record and Report Employee Web Activity a. Select Configuration > Access Logging > Logs > General Settings. b. Select main as the Log type. c. Verify that the Log Format defaults to bcreportermain_v1. d. If it does not, select main and click Apply. 3. If you require reports for streaming media traffic, repeat Step 2. Select streaming as the Log and verify that the default is bcreporterstreaming_v1. 4. Begin Access Log recording. a. Select Configuration > Access Logging > General > Default Logging. b. Select Enable Access Logging and click Apply. Next Step: Upload Access Logs to the Reporter Server Upload Access Logs to the Reporter Server Configure the ProxySG appliance to upload the Access Log files to the server that you have dedicated for Blue Coat Reporter. Tip Consult your planning form if you have one, or if someone in your organization provided you with one. 6
1. Log in to the ProxySG Management Console. 2. Select Configuration > Access Logging > Logs > Upload Client. 3. Configure the FTP upload client for the main (bcreportermain_v1) access log. a. From the Log drop-down, select main. b. In the Upload Client field, select FTP Client. Note: Do not select Blue Coat Reporter Client. This client is for direct stream of data into Reporter, which does not retain the raw access logs. For more information, consult the Blue Coat Reporter Initial Configuration Guide. c. Click Settings. The Management Console displays the FTP Client Settings dialog. 4. Enter the access credentials to the FTP server that stages the Reporter logs. 7
Record and Report Employee Web Activity a. Enter the Host server's IP address. Only change the Port if it uses a different one. b. Enter the Path, which is the destination of the log files. For example, create a folder that indicates where this gateway ProxySG is located or what set of users it includes. This helps you with folder management on the server. c. Enter the username required to access the server. d. If a password is also required, click Change Primary Password. In the Change Primary Password dialog, enter the credentials and click OK. e. Click OK. 5. If you have a backup staging server configured, repeat Steps 3 and 4; in Step 4, select Alternate FTP Server. 6. In the Transmission Parameters area, select the Save the log file as: gzip file option. Blue Coat recommends this option, as most deployments process multiple gigabytes (Gb) of data. 7. Click Apply. 8. Test the FTP connection. a. In the Upload Client area, click Test Upload. b. In the Management Console, select Statistics > Access Logging > Upload Status. c. Verify upload client connection or troubleshoot the connection as necessary. d. After you verify the connection, delete the test file. 9. To begin uploading the log files to the Reporter staging server, select Configuration > Access Logging > Logs > Upload Schedule. 8
a. From the Log drop-down list, select main. b. (Optional) If employee-generated traffic has already occurred, click Upload Now to FTP the logs that are currently stored to the Reporter server. This allows you to immediately set up and test the Reporter log source. c. Select to upload the logs periodically. d. Specify when the ProxySG appliance initiates the FTP upload. Blue Coat recommends once per day during a time when employees are least likely to be generating traffic. e. Click Apply. 10. If you are also sending streaming media access logs, repeat Steps 3 through 9. In Step 3a, select streaming as the Log. Next Step: Create a Reporter Log Source. Create a Reporter Log Source This topic is a sub-set of the initial Blue Coat Reporter initial configuration process. It demonstrates how to configure a database and log source, which reads access logs uploaded from a gateway ProxySG appliance to the Reporter staging server. 9
Record and Report Employee Web Activity This procedure assumes that you have installed the Reporter application and have admin privileges. If you require the full installation procedure, consult the Blue Coat Reporter 9.4 Initial Configuration Guide. 1. Log in to the Blue Coat Reporter application. 2. On the General Settings page, select Data Settings > Databases. 3. Click New. Reporter displays the Create New Database wizard. 4. On the initial Set Type screen, select ProxySG (main); click Next. 5. Enter a Database Name. A meaningful name aids with account management. For example, if this database will build from Access Logs from a specific region or location, enter a related name. Click Next. 6. Specify the Log Sources. a. Click New Log Source. The wizard switches to the Create New Log Source page. b. The Set Type log source option depends on where you installed the Reporter application. If you installed Reporter on the same server as the staging server, select Local File Source. If you installed Reporter on a separate machine, select FTP Server Source. Click Next. c. Enter a Log Source Name. Again, a meaningful name helps with management. d. The Set Location page varies depending on whether you selected Local or FTP source. For the Local File Source, browse to the folder location. For the FTP File Source, enter the FTP server information. 10
Click Next. d.1. Enter the Hostname or IP address of the server and the Port number. d.2 Enter the Username and Password required to access the server. d.3 Enter the Directory Path where the Access Log folder(s) exist. e. On the Set Log File Check Frequency page, specify how often Reporter checks for Access Log files that it has not yet processed. Select Custom Schedule. Use the drop-down to select a periodic time frame. If you are performing a test, select the Once option and set a time for a few minutes from now; or, select Periodic and set for every few minutes. When you are satisfied with testing, you can return to this log source and edit the schedule. Otherwise, select how often the check occurs. For example, set Reporter to check every day during non-use hours. 11
Record and Report Employee Web Activity If you leave the Default option selected, you can configure a global schedule for all sources in the database. Step 7 below describes what occurs; for now, click Next. f. On the Set Post Processing Action page, specify what happens to the Access Log files after Reporter processes them. Rename: Append '.done' to filename After Reporter processes a log file, it adds.done to the existing.log or.gz suffix. When you browse the directories with a file viewer, this is how you know when files have been processed. Be advised, if you delete the.done suffix, Reporter will reprocess the log file. Move to folder After Reporter processes a log file, the file moves to the specified directory (or subdirectory tree if Process Subdirectories was selected on the Set Location wizard page). Should you ever require a reprocessing of log files, you can copy the files back to the directory. Remove: Delete log file After Reporter processes a log file, the file is deleted. Select this option if you are certain you will never have the need to process those log files again. Click Done. 7. The wizard returns to the Set Log Sources wizard screen and displays the new log source. At this time, you can add another log source; for example, you also configured the ProxySG appliance to upload a streaming media data Access Log and you want the data from those logs to be added to this database. Click New Log Source and repeat Step 6. Click Next. Note: Notice the Default check for new log files option. If you do not specify a custom schedule for how often Reporter checks for new logs to this specific log source (Step 6.e), the check occurs according to this default schedule. The per-log source schedules override this default. 8. To force Reporter to stop generating report data for dates beyond a specified time frame, select Expire database data older than, specify when data expires, and select the Frequency (when Reporter checks the database). For example, if the database contains log files processed with March 1st as the earliest date, the setting is 30 days, and the current date is April 1st, Reporter no longer generates and displays report data for March 1st. (Reporter deletes the data from the database.) Click Next. 9. For the Set Directory options, the defaults are sufficient. Click Done. Reporter displays the new database and log source information and begins to build the database (assuming that you have uploaded, unprocessed Access Log files in the specified directories. Next Step: View User Web Activity Reports View User Web Activity Reports After you configure the ProxySG appliance to record and send Access Logs to the Reporter server and configure the Reporter log source, you can view various reports (following a period of browsing by users and after the first scheduled Access Log transfer). 12
You can click the Help (?) button on the Reports page to display brief descriptions of each report. The following are of interest. User Behavior Web Browsing per Category When an employee requests (browses) to a website, that site is rated and matched to a category (for example, news/media, business/economy, mature, and so on). This report lists all of the website categories that were browsed by employees, sorted by the highest Page Views per category. Intended audience: HR; persons who are interested in viewing individual user Web browsing activity. Use Case You review the report and notice that the Shopping category results are large, which indicates that employees are consuming too much time on non work-related websites. The person who manages Web access policy can adjust the policy or provide a coaching mechanism for employees. Web Browsing per User This report displays every user reported in the processed access logs who requested Web content, sorted by the total number of requested pages. Security Intended audience: HR; persons who are interested in viewing individual user Web browsing activity. Use Case In reviewing this page, you notice two users brian.underwater and christopher.lewis requested a noticeable higher number of websites than other users. Their position within the enterprise might warrant such activity, but they might also need to be coached on company Web use policy. Blocked Web Sites This report lists the websites that users attempted to access but were denied by Web-use policies. By default, Reporter lists each site ranked by the highest number of requested Web pages. Intended audience: IT; persons who are responsible for creating policy that enforces the company's Web use policies. Use Case If you have created and installed policies that block questionable website categories that are not deemed appropriate for your particular enterprise, you might on occasion generate this report to review what specific sites are constantly requested by users (and subsequently denied). The constant presence of specific inappropriate website requests might require a severe coaching mechanism or other communicated bulletin to the employees. Potential Malware Infected Clients This report lists all client IP addresses that might be infected by malicious content. This data is derived by the URLs requested by each client. By default, Reporter lists each IP address, sorted by the number of requests to possible URLs that are known sources of malware/spyware. Intended audience: IT; security team members can use this report as a to-do list to visit infected machines and run anti-malware cleaners. Use Case You have discovered that user browsing activity is allowing malware to infiltrate your network and you want to see how many users are responsible. For example, one user may be responsible for 33% of the malware invasion. For further analysis, you apply a filter to review the sites that contained the potential malware. The filtered report displays the top malware-source sites, ranked by HTTP requests. 13
Record and Report Employee Web Activity Web Application Reports Web Application Detailed Report The data in this report displays detailed information on Web applications (social networking, blogging, tagging) sites based on the page view count. You can change the number of records to be displayed by changing the filter conditions in Report Options. Intended audience: IT; persons who are responsible for creating policy that enforces the company's Web use policies. Use Case Web applications are critical for successful business, but can also introduce many time-wasting elements, potential security risks, and excessive bandwidth use. IT wants to monitor who is using what applications and adjust policies accordingly, such as allow an application but block file downloads or games. 14
Access Logging Troubleshooting Why is the ProxySG uploading logs so frequently? 15 Why is the ProxySG uploading logs so frequently? Problem: The ProxySG appliance is uploading logs more frequently than expected. Resolution: Access Logs accrue on the ProxySG appliance hard drive and eventually reach storage capacity. For the Access Logging solution in this WebGuide, Blue Coat recommends configuring the ProxySG appliance to trigger an upload ahead of schedule when data reaches a specified amount of megabytes. 1. Select Configuration > Access Logging > General > Global Settings. 2. The default Global Log File Limits values will vary depending on the capacity of each gateway ProxySG model. Consult the sizing guide for information. To trigger a log upload rather than halt all logging, the second value must be lower than the first value. 3. Click Apply. Tip To prevent the Access Logs that do not have a configured upload client from triggering an early upload threshold, edit the default logs for each protocol that do not require uploading. Set them to <None> from the Configuration > Access Logging > Logs > Upload Client tab. 15