Version 2.4.2 connect. code. create.
This edition of the System Administrator s Guide refers to version 2.4.2 of This document created or updated on March 3, 2014. Please send your comments and suggestions to: Black Duck Software, Incorporated 8 New England Executive Park, Suite 221 Burlington, MA 01803 USA. Copyright 2014 by Black Duck Software, Inc. All rights reserved. All use of this documentation is subject to the license agreement between Black Duck Software, Inc. and the licensee. No part of the contents of this document may be reproduced or transmitted in any form or by any means without the prior written permission of Black Duck Software, Inc. Black Duck, Know Your Code, and the Black Duck logo are registered trademarks of Black Duck Software, Inc. in the United States and other jurisdictions. Black Duck Suite, Black Duck Code Center, Black Duck, Black Duck Protex, Black Duck Export, Black Duck Transact, and Koders are trademarks of Black Duck Software, Inc. All other trademarks or registered trademarks are the sole property of their respective owners.
Table of Contents TABLE OF CONTENTS 1 Introduction... 1 1.1 architecture... 1 1.2 process... 2 1.3 Supported source control management (SCM) clients... 2 1.4 Understanding what indexes... 3 1.5 Logging in... 3 2 Managing Users... 5 2.1 Understanding user roles... 5 2.1.1 Understanding permissions... 6 2.1.2 Creating a new user role... 7 2.2 Creating a new user account... 9 2.2.1 Adding roles to a user... 11 2.3 Creating user groups... 14 2.4 Using LDAP or Active Directory authentication... 17 3 Managing Projects... 18 3.1 Understanding how Code Site connects to repositories... 18 3.2 Understanding projects... 18 3.3 Creating a project... 18 3.4 Configuring the connection to a code repository... 21 3.4.1 AccuRev... 23 3.4.2 Bazaar... 23 3.4.3 Borland StarTeam... 23 3.4.4 CVS... 23 3.4.5 File system adapter... 24 3.4.6 Git... 24 3.4.7 IBM Rational ClearCase... 25 3.4.8 Mercurial... 25 3.4.9 Microsoft Team Foundation Server (TFS)... 25 3.4.10 Perforce... 25 3.4.11 Subversion... 26 3.4.12 Zip archive adapter... 26 3.4.13 and other version control systems... 27 Page i
Table of Contents 3.5 Modifying a project... 27 3.6 Understanding project access... 28 3.6.1 Restricting who can search a project... 28 3.6.2 Adding team members to a project... 30 3.6.3 Hiding a project from search... 31 3.7 Understanding project crawling... 33 3.8 Understanding indexing schedules... 33 3.8.1 Setting the global indexing schedule... 33 3.8.2 Set the indexing schedule for a specific project... 34 3.8.3 Excluding a project from indexing... 35 3.8.4 Forcing to index a project... 36 3.9 Viewing project history... 37 3.10 Deleting a project... 38 4 Configuration Settings... 40 4.1 Configuring indexing settings... 40 4.1.1 Creating a global indexing schedule... 40 4.1.2 Configuring indexing to ignore directories and files... 41 4.2 Configuring SCM connection timeout... 41 4.3 Configuring email settings... 42 4.4 Configuring proxy settings... 42 4.5 product registration... 42 4.5.1 Activating... 43 4.5.2 Reactivating... 44 4.6 Configuring updates... 45 5 Managing the Server... 48 5.1 Disk space requirements and usage... 48 5.2 Startup, configuration, and log Files... 48 5.3 Stopping and starting the Tomcat server running on Linux... 49 5.4 Stopping and starting the Tomcat server running on Windows... 50 5.5 Configuring password strength... 52 5.6 Enabling the consistency checker... 55 5.7 Backing up the database... 57 5.7.1 Creating a database backup on Linux... 58 Page ii
Table of Contents 5.7.2 Creating a database backup on Windows... 59 5.7.3 Restoring the database on Linux... 62 5.7.4 Restoring the database on Windows... 63 APPENDIX A Data Gathered by... 67 APPENDIX B Using and Configuring LDAP or Active Directory... 68 B.1 Configuring to use LDAP or AD... 68 B.2 Importing and synchronizing users... 71 B.3 Granting roles to groups of users... 72 B.4 LDAP configuration examples... 74 B.4.1 Authentication searches directly below a domain component node... 74 B.4.2 Authentication searches restricted to a branch node... 77 Page iii
Preface PREFACE Target audience This manual is intended for the administrator responsible for configuring and managing Black Duck. Most of the setup described in this book is done using the Admin section of the user interface. You must log in to using an account that has been granted a role with administrative privileges to access this area. Related documents The documentation for consists of: Title File Description Installation Guide InstallationGuide.pdf Detailed installation instructions for. Release Notes RelNotes.pdf Description of what has changed from the previous release of. System Administrator s Guide AdminGuide.pdf Describes administrative and configuration tasks in. User s Guide UserGuide.pdf Describes how to use the search web page. SDK Code Examples Customizing the Code Sight User Interface CodeSightSDK_Examples.pdf Customizing_Code_Sight_UI.pdf Describes how to install the SDK and contains code examples. Note: The SDK documentation is available with a standard license, but the SDK requires a special license to run. Describes how to enable UI customization and provides some general examples of the types of customization that are commonly requested. The documentation for is included in the top-level directory of the installation media. After installation, the PDF guides are located in C:\Program Files\Black Duck Software\CodeSight\doc. Note: This directory may vary depending on your operating system. You can also access the PDF guides from links on the Welcome page in the online help. Page iv
Preface Customer support There are three versions of available, differentiated by license: A free unregistered version, which lets you index 200,000 lines of code. A free registered version with a trial license with limits set forth at: http://www.blackducksoftware.com/go/codesight_additional_terms. Support for the free versions is provided using the online support forum at: http://forum.blackducksoftware.com/ An enterprise version, which allows for potentially unlimited indexing (limited per your license agreement). Support for the enterprise edition is provided through the online support forum and through the regular Black Duck Technical Support channels: Email: support@blackducksoftware.com Phone: +1 781.891.5100, ext. 5 Fax: +1 781.891.5145 Page v
Introduction 1 INTRODUCTION A significant portion of application development involves a process of find, copy, paste, and integrate. This process can be greatly accelerated when you can find and download existing source code that has already been debugged, tested, and approved. makes it easy for software developers to find code in source code repositories that they have access to. Once you have found a useful file or snippet, download it for use in a new project. Tip: is intended to work with your company s repositories and local directories. While you could also crawl and index any open source projects you found on the web, if you are interested in doing this you should check out Black Duck Ohloh Code. This is a free, hosted version of the search engine with an index of billions of lines open source code. 1.1 architecture Black Duck is a web application. A implementation consists of several different components that work together: Database: uses an industry-standard PostgreSQL database. installs a dedicated local instance of PostgreSQL for its own use. stores local copies of the files downloaded from your repositories. Server: uses an Apache Lucene Solr search server that runs within an Apache Tomcat application server. User interface: uses a web-based user interface for both searching (Search home page) and configuration and administration (Admin pages). Users with Administrator privileges can create and manage users, define projects and code locations, and schedule crawling, indexing, and publishing. In addition, has a software development kit (SDK). The SDK lets you integrate development into your native development processes. It provides API access to development features and services. Page 1
Introduction 1.2 process In order to configure and troubleshoot, it is helpful to understand the Crawl/Index/publish cycle that employs. The process flow is as follows: 1. Your system administrator installs and any source control management (SCM) clients needed to access your repositories. 2. Your system administrator configures to access your company s code repositories. This involves the following: a. Creating projects. The project definition includes connection strings, which tell how to access your code from one or more SCM repositories. b. Setting up a schedule for how often should revisit the projects to look for new or updated files. 3. connects to your repository via an SCM adapter and crawls the source code in your projects. 4. records the source file metadata in the database and generates a Solr index. 5. checks the index for consistency and merges the index with any other project indexes. 6. publishes the merged index, making the new project data available for user searches. 7. Developers access the search page (http://<codesightserver>) to search your repositories for code. 8. repeats the Crawl/Index/Publish cycle based on the schedule specified by the system administrator. 1.3 Supported source control management (SCM) clients supports the following open source and commercial SCM clients: AccuRev Bazaar Borland StarTeam CVS Git IBM ClearCase Mercurial Microsoft Team Foundation Server Perforce Subversion Note: File System Adapter and Zip are built into and do not require a client. Page 2
Introduction 1.4 Understanding what indexes As examines your projects, it indexes the following types of files: ActionScript Ada ASP ASP.NET Assembly Boo C C# C++ Cobol ColdFusion Delphi Eiffel Erlang Fortran F# Groovy Haskell Java JavaScript JSP Lisp Lua Mathematica Matlab Modula ObjectiveC Ocaml Perl PHP Prolog Python REBOL Rexx Ruby Scheme Smalltalk SQL Tcl VB VB.NET XML For C, C++, C#, Java, JavaScript, Ruby, and Python sources, provides advanced parsing. This allows filtering to definitions and provides a special outline view. For the other languages it indexes, provides syntax highlighting where applicable. Other non-source file types, including.log and.html, are also available for basic searching, but they are not parsed and indexed for source content. 1.5 Logging in No login is required for search in or run reports. However, to create and index projects, you must log in with a privileged account. To log in to 1. Launch a browser and navigate to the URL for your server, which should be sometime similar to the following: http://<codesightserver> Page 3
Introduction 2. Select the Login link. 3. Enter your Username and Password and click Login. Note that passwords are case-sensitive. 4. Select the Admin link in the upper right corner. displays the administrative area, which is where you define how gets to the code it needs to index. The default page view is the Progress tab, which shows the current status of indexing on this machine. The status may show as RUNNING if the machine has been configured to run the indexer continuously, always looking for new projects. Alternately, indexing can be run at scheduled intervals. Page 4
Managing Users 2 MANAGING CODE SIGHT USERS Depending on your company policies for allowing access to code repositories, you can configure to allow or restrict user access to your projects and files: Any user can search and access projects and files (no user ID required). Only logged in users can search and access projects and files. Only named users can search and access files for a project. You configure the list of named users for each individual project. Users are allowed access to projects based on their user roles. Roles are either system-wide (overall roles) or project-level (project roles). 2.1 Understanding user roles You use roles to grant user permissions to features and actions within. has two different types of roles: Overall roles grant global permissions that are applicable throughout. Project roles grant permissions that apply only to an individual project. comes with three pre-configured user roles with varying permission levels. When you create a new user, you assign one or more roles to the user ID to grant the user permissions to various pages and actions within. The default overall roles are: Administrator Indexer Project Searcher (This is the default role.) The pre-configured roles are locked and cannot be edited or deleted. You can, however, create custom roles and assign them to users as appropriate for your business needs. You can create both custom overall and project-specific roles. Page 5
Managing Users 2.1.1 Understanding permissions Table 1 shows the combinations of permissions in the default system roles. Table 1 Permission matrix Project Role Overall Role Can read registration settings Can change registration settings Project Searcher Can read user-management configuration data X X Can have update access to user management configuration data Can create user management configuration data Can delete user management configuration data Can access the Admin area X X Can read application settings X X Can change application settings X X Can create a project X X Can start/stop the index engine X X Can publish (index data for projects) X X Can see the team members on a project X X Can assign team members to a project X X Can search the code of a project X X X Can read project data X X Can update project data X X Can delete a project X X Can index a project X X Can schedule a project (indexing) X X Can show/hide a project X X Indexer X X X X X Administrator Most of the permissions are quite specific and allow access to individual controls within. One of the more powerful permissions is the ability to create user management configuration data. This permission allows you access such as creating users and giving them roles. This should only be granted to System Administrators. Page 6
Managing Users 2.1.2 Creating a new user role You may want to create new, custom roles with different combinations of permissions than the default system roles. In general, when combining permissions to create a role: If you have delete access, you should also have read, update, and create access. If you have create access, you should also have read and update access. If you have update access, you should also have read access. Read access can stand alone. Caution: Many pages within are tied to various permissions. If you turn these permissions off, you cannot see the pages. If you are creating custom roles to use instead of the default roles, make sure that your new roles provide access to all of the existing permissions and do not leave any holes, that is, a permission not covered by a role. For example, if you remove read user management configuration data from your own role, you cannot see the User section. In that case, you would not be able to go to the Admin Users Roles page to turn the permission back on. To create a new user role 1. Log in to as a user with administrator privileges. If you have not yet created an administrator account, you can use the pre-configured default account: User name: sysadmin Password: blackduck 2. Select the Admin link in the upper right corner. 3. Select the Users Roles tabs. 4. Click Create Role. The Create a Role dialog box opens. Page 7
Managing Users 5. Enter a name for the role. 6. Enter a description for the role. 7. Specify whether the role is overall (system-wide) or will be specific to a particular project. 8. Select one or more check boxes to set permissions for the role. 9. Click Save. Page 8
Managing Users To modify a user role 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Users Roles tabs. 4. In the Roles table, highlight the role to select it. 5. To delete the role, click Delete Role. 6. To modify the role, edit the role and click Save. 2.2 Creating a new user account Notes: includes a default system administrator s account. Black Duck recommends changing the default password immediately after installation. - The default administrator account cannot be deleted or disabled, and can be used to add all of the other user accounts you will need. - If you are using an LDAP server, see Appendix B Using and Configuring LDAP or Active Directory. A user with Administrator privileges can create new user accounts. Each account includes descriptive information about its owner. Table 2 shows the data contained in a user record. You are only required to enter a user name and password in order to create a user record. Note that the account information does not indicate what project the person is working on. That kind of information is considered the user s role, and is defined in a subsequent step. Table 2 User Account Fields Field *Username *Password *Password Confirmation First Name Last Name Description A unique name for this person to use when logging in. The name is not case-sensitive. The name is attached to history records of the user s actions within. Passwords are case-sensitive, and cannot be blank. They do not expire. By default, does not enforce any restrictions on user names or passwords. For information about enforcing password requirements, see Configuring Password Strength on page 52. This field needs to match the password entered above. The user s full name is useful for identifying an individual by more than just their user name. displays the user s name in the Users table on the Users Users tab and in dialog boxes that let you select a user. (see previous) Page 9
Managing Users Field Email Address Job Title Phone Location Active Description The email address is useful as a reference. It is not currently used within the product. The job title for a user is useful as a reference. displays the job title in the Users table on the Users Users tab. The phone number is useful as a reference. It is not currently used within the product. The location is shown on the list of users and is useful as a reference. It is not currently used anywhere else within the product. An account must be set to active to enable logins. Tip: First and last names and job title are only displayed in the Users section. It is useful to add the user s first and last name, so that when you are adding users to a group you can identify the individual by more than just the user name. To create a new user account 1. Log in to as a user with administrator privileges. If you have not yet created an administrator account, you can use the pre-configured default account: User name: sysadmin Password: blackduck 2. Select the Admin link in the upper right corner. 3. Select the Users Users tabs. 4. Click Create User. The Create a New User dialog box opens. Page 10
Managing Users 5. Enter a unique user name for the account. 6. Enter a password for the user. 7. Optionally, enter any additional information that you want to record about the user. 8. Click Save. By default, all user accounts have the Project Searcher role assigned to them. Typically, users have distinct needs depending on their project assignments, and therefore you can assign multiple overall and project roles to each user. 2.2.1 Adding roles to a user After you create an account, you must add roles to the account that permit access to projects and repositories. You can view the roles assigned to a user on the Users Users Roles tab. Note: All users have the Project Searcher role by default. This role is not displayed on the Users Users Roles tab. See Figure 1. Page 11
Managing Users Figure 1 New user account with no roles To assign overall roles to a user ID 1. Log in to as a user with administrator privileges. 2. Select the Admin link. 3. Select the Users Users tabs. 4. In the Users table, click the user name to select it. 5. Select the Roles subtab. Page 12
Managing Users 6. Click Add Overall Role. 7. Select one or more roles. 8. Click Save. Note: Any time you change a user s role, they need to log out and log back in again for the change to take effect. To assign project roles to a user ID 1. Log in to as a user with administrator privileges. 2. Select the Admin link. 3. Select the Users Users tabs. 4. In the Users table, click the user name to select it. 5. Select the Roles subtab. 6. Click Add Project Role. Page 13
Managing Users 7. Select one or more roles. 8. Select one or more projects. Note: For project-specific roles, you must indicate both the role and the project name. 9. Click Save. Note: Any time you change a user s role, they need to log out and log back in again for the change to take effect. 2.3 Creating user groups A user group is an easy way to configure multiple user accounts with the same roles. To create new user group 1. Log in to as a user with administrator privileges. 2. Select the Admin link. 3. Select the Users User Groups tabs. 4. Click Create User Group. Page 14
Managing Users 5. Enter a group name. 6. Click Save. The next step is to assign permissions to the group by assigning roles. To assign users to a user group 1. On the Users User Groups tab, click a user group to select it. 2. If necessary, in the bottom half of the page, scroll until you can see the Group Members table. 3. Click Add Member. 4. Select one or more members to add to the group. 5. Click Save. Note: Any time you modify users roles, they must log out and log back in again to see the changes take effect. To assign permissions to a user group 1. On the Users User Groups tab, highlight a user group to select it. 2. If necessary, in the bottom half of the page, scroll until you can see the Roles tables. Page 15
Managing Users 3. To add system roles to the group: a. Click Add Overall Role. b. Select one or more roles. c. Click Save. 4. To add project roles to the group: a. Click Add Project Role. b. Select one or more roles. c. Select one or more projects. d. Click Save. Note: If users already belong to the group, they will need to log out and log back in to see the changes take effect. Page 16
Managing Users 2.4 Using LDAP or Active Directory authentication supports authentication using its own standard username and password scheme. It also supports using lightweight directory access protocol (LDAP) or Active Directory (AD). See Appendix B Using and Configuring LDAP or Active Directory for details on configuring this feature. Page 17
Managing Projects 3 MANAGING PROJECTS In order for users to be able to search your code, you must configure to know where your source files are, and how it can access them. The basic entity within is the project. A project describes the location of your source files, which users can access them, and how often the files should be crawled and indexed. 3.1 Understanding how Code Site connects to repositories Your source files likely reside in a remote SCM repository. Within, you create a project, which points to locations within your repositories. The system where the server is installed must have the client software installed for the repositories and that client software must be in the path visible to the process. 3.2 Understanding projects A project is the basic entity within. In a project record only requires a name and the connection information for the repository. You can enter multiple source code locations for a single project. These can be separate subdirectories, branches, or completely different source control management repositories. During the crawling phase, visits each location (in the order you enter them) and retrieves the files. visits each repository as directed by the project s overall schedule. Note: You can also create a project with no source code. 3.3 Creating a project Tip: Always enter useful information in the Project Description field, because this text is searchable. To create a project 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. Page 18
Managing Projects 3. Select the Projects tab to view the projects already configured on this system. 4. Click Create Project. The Project Setup dialog box opens. 5. Enter a unique name for the project. Note that the name and description are both searchable. 6. Enter the connection details for the repository containing the project. Page 19
Managing Projects Note: For information about configuring and connecting to SCM clients, see Configuring the Connection to a Code Repository. If your repository does not require a username and password, you can leave those fields blank. 7. Optionally, you can enter multiple locations for where your source code is located. a. Select the Add Code Location link. b. Enter the connection details for the repository containing the project. Tip: Many repository types allow browsing for a specific branch or tag. Use the Test Connection button to determine your proper connection string, and then use Browse to find the subdirectory you want. c. To add additional code locations, repeat steps a through b for each code location. 8. Optionally, in the Advanced Settings section, you can restrict the search visibility (who can search the project). Page 20
Managing Projects 9. Optionally, in the Advanced Settings section, you can change the indexing schedule (schedule of how often the project is crawled by ). 10. Click Save. 3.4 Configuring the connection to a code repository The system where the server is installed must have the client software installed for the repositories, and that client software must be in the path visible to the process. Note: uses the PATH environment variable to look for SCM clients. Make sure your SCM client can be found using the system path, and not your user path. has built-in support for several different SCM systems, as well as the File System Adapter that can be used with almost any version control system. The Settings SCM Settings tab (see Figure 2) shows a list of the clients that has detected on your system. The File System and Zip adapters are built into. Page 21
Managing Projects Figure 2 SCM settings Tip: The link at the bottom of the tab takes you to the topic in the online Help system where you can find additional information about each supported repository client. SCM repositories used with have several common aspects. Depending on the SCM type, these fields may be separate or may be combined into the connection string. The connection string for a repository is similar to the command you would use to interactively connect to the repository. In order to connect to a code repository, you will need the following information: Username/password you use to access the repository Connection string of the SCM client Note: The connection string for a repository is similar to the command you would use to interactively connect to the repository. Tip: The online Help system includes examples for connecting to an open source repository of each supported type. Branch or tag where your source code is located within the repository Page 22
Managing Projects 3.4.1 AccuRev Tips: - Always make sure the AccuRev executable is in your path. - Visit http://www.accurev.com for more information. - AccuRev requires both a valid server connection string and a valid stream name. If the stream name is not provided, the test connection will work, but crawling will fail. The standard template connection string for an AccuRev repository looks like the following: Username: myname Password: ****** Stream: Host:Port Branch or Tag: /Project[view/folder] 3.4.2 Bazaar Tip: Download Bazaar and make sure it is in your path. The Bazaar connection string includes the server location and project branch in the following format: http://host.com/path/to/my/branch Example Connection String (anonymous user): http://bzr.savannah.gnu.org/r/emacs 3.4.3 Borland StarTeam Tips: - Always make sure the Borland StarTeam executable is in your path. - Visit http://www.borland.com/us/products/starteam/ for more information. The standard connection information for a Borland StarTeam repository looks like the following: Username: user Password: ***** Connection String: host:49201/project[view/folder] 3.4.4 CVS Tip: Download CVS for All Platforms and ensure cvs.exe is in your path before starting the Code Sight service. Generic non-anonymous connection string :pserver:username:password@serverip:/path/cvsroot Apache connection string :pserver:anoncvs:anoncvs@cvs.apache.org:/home/cvspublic Berlios connection string :pserver:anonymous:@cvs.projectname.berlios.de :/cvsroot/ Page 23
Managing Projects Mozilla connection string Enter the following on a single line for the Connection Template field on the Add/Edit Repository page: :pserver:anonymous:anonymous@cvs-mirror.mozilla.org:/cvsroot When adding a project that uses this repository, specify the project name and path. You can use the Browse button or enter the values directly. Novell Connection String :ext:anonymous@forgecvs1.novell.com:/cvsroot/. Note: The period (.) at the end of this template is used to represent all modules for the given CVS root. 3.4.5 File system adapter The file system adapter enables to index source code stored on a local machine. The file system adapter acts as a universal adapter, allowing you to use with any SCM application. Simply access your repository and retrieve the files to a local directory, then point at the local files. The connection string when using the file system adapter is the absolute path to the top-level directory, such as: C:\MySourceCode\ Note: When using the adapter for this purpose, you are responsible for making sure that the source code in the path specified is synchronized with the repository server. 3.4.6 Git Tip: You do not need to download a Git client. This functionality is included in your installation. Due to the decentralized nature of the Git SCM system, you cannot use the Browse feature to list projects in the repository. Also, you cannot use the Test Connection button unless you include the full path to the project in the repository connection string. For example, consider that you want to index the following project: git://github.com/mislav/will_paginate You could set the repository s Connection String to: git://github.com/mislav/ and set the project s Project Path to: will_paginate However, this method would not allow you to test the connection. A better practice is to include the Page 24
Managing Projects full project path in the connection string. 3.4.7 IBM Rational ClearCase Tip: Download IBM Rational ClearCase or the IBM Rational ClearCase Remote Client and make sure it is in your path. Template: PathToView\VobName$SubPath Example: c:\views\someview\sources$initialcomponent\ Note: The adapter assumes PathToView contains a valid and configured ClearCase View. Using the Remote Client: Username: username@domain.com Password: ****** Connection String: C:\view\myView\myVOB$mycmpnt\ Or, you can combine the fields into a single connection string: --username user@bds.com --ser http://localhost:12080/teamweb/services/team --pas changeme M:\MyFirstUCMView\ 3.4.8 Mercurial Tip: Download Mercurial and make sure it is in your path. Template: http://hg.address Example Connection Template: http://hg.savannah.gnu.org/ Example Project Path: hgweb/octave/ 3.4.9 Microsoft Team Foundation Server (TFS) Tips: - Always make sure the TFS executable is in your path. - A TFS client cannot go beyond 259 characters. - Go to Team Foundation Information for more information. The standard template connection string for Team Foundation Server looks like the following: Template: tfs://username:password@server$/ UNC Server: tfs://username:password@\\myserver$/ourproject Web Server: tfs://username:password@http://myserver:8080$/ourproject/path The password component is optional. Server specification is typically a UNC path or http path with a port specified following a colon (as in http://myserver:8080). TFS uses a web services base, so users having poor luck or inconsistency with UNC paths may find using an http specification easier. 3.4.10 Perforce Tip: Download Perforce and make sure it is in your path. Page 25
Managing Projects Connection strings for Perforce must have the username, password, and server location specified in separate fields. For example: Username: guest Password: <blank Connection String: public.perforce.com:1666//public/perforce/utils/aegis/ 3.4.11 Subversion Tip: Download Subversion and make sure it is in your path. Generic Subversion connection string svn://svnserver.mycompany.com/trunk/ If you are connecting to a system that allows anonymous logins, you may be able to omit the username and password fields. Apache connection string http://svn.apache.org/repos/asf/ Code.Google.com connection string http://projectname.googlecode.com/svn/trunk/ Or, depending on your specific project, you may need to specify a branch instead. In this case, use the /tags qualifier instead of /trunk: http://projectname.googlecode.com/svn/tags/ For example, a sample connection string is: co http://oreguile.googlecode.com/svn/trunk/ Often, projects on Google Code say that to download them, you must pass projectname-read-only as a parameter. This is not necessary when using to access the project files. 3.4.12 Zip archive adapter The Zip archive adapter is useful for looking inside.zip archive files. 1. Create a new project using Zip as the SCM Client type. 2. For the Connection String, enter the absolute path to a top-level directory containing your Zip files. For example: /etc 3. Use the Browse button to find and select your ZIP files. Tips: The file system adapter does not open.zip archive files, so you may need to use both repository types to fully index your project. Also, the Zip adapter does not handle nested zip archives, so these would need to be expanded manually. The Zip adapter only expands the first level. Page 26
Managing Projects 3.4.13 and other version control systems can be used with other version control systems as well. When integrating with other version control systems, use the file system adapter. The file system adapter enables indexing, but it does not ensure that the local source code cache is up-to-date. Note: When using the File System Adapter, the system administrator must make sure that the local source code folders are kept current by creating the necessary scheduled scripts or commands. 3.5 Modifying a project After a project has been created, you can edit the information and schedule. To modify a project, complete the following steps: 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Projects tab to see the projects already configured on this system. 4. Highlight a project name to select it. 5. In the lower half of the screen, select the Setup subtab. 6. Click Edit Project. The Edit Project Setup dialog box contains the same fields as the Create Project Setup dialog box. Page 27
Managing Projects 7. Modify the project definition as appropriate. 8. Click Save. 3.6 Understanding project access You can set up your project for user-based search access. Rather than allowing everyone to search the project code, you can restrict access to a list of named users or user groups. You can also hide a project from all searchers. 3.6.1 Restricting who can search a project You can restrict who can search a project when you first create it or you can modify this setting for existing projects. To restrict project search to named users/groups, complete the following steps: 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Projects tab to see the projects configured on this system. 4. Select the checkbox for the project in the Projects table. 5. From the More actions menu, select Restrict for search. Page 28
Managing Projects 6. Alternately, you can Select the Setup tab and edit the project to change the Search Visibility setting to Restricted. The project will only be visible to users listed as a Team Member on the project. Page 29
Managing Projects 7. Add users to the list of team members for the project. For instructions, see Adding Team Members to a Project. 3.6.2 Adding team members to a project When you set the Search Visibility setting for a project to Restricted, the only users who can search the project are users or groups listed on the Team Members tab for the project. To add team members to a project, complete the following steps: 1. On the Projects tab, highlight the project name to select it. 2. In the lower half of the screen, select the Team Members tab. 3. To add a user to the project, click Add User. The Add User to Project dialog box opens. Page 30
Managing Projects a. Highlight a user name to select it. b. Select one or more roles for the user. c. Click Add User. The change takes effect immediately. 4. To add a user group to the project, click Add User Group. a. Highlight a user group to select them. b. Select one or more roles for the user group. c. Click Add User Group. The change takes effect immediately. Note: You can also use the Users tab to add the project to the user s record or the User Groups tab to add the project to a user group s record. 3.6.3 Hiding a project from search At certain times it may be necessary to make a project non-searchable while still maintaining the option to index it later. Or, you may want to freeze the current state of the project and disable indexing. Both of these cases are possible. To hide project from all searchers, complete the following steps: 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Projects tab to see the projects configured on this system. 4. Select the checkbox for the project in the Projects table. 5. From the More actions menu, select Hide for search. Page 31
Managing Projects 6. Alternately, select the Setup tab and edit the project to change the Search Visibility setting to Hide. The project will be hidden from all users searching. Page 32
Managing Projects 3.7 Understanding project crawling Once you have defined a project, the next step is to have examine the source files (also known as crawling ) and then index the project. During this process, retrieves any new or changed files from the SCM repository and performs a special analysis pass on them based on their file type. Then, it publishes the new index, making the project searchable for all users. The system administrator specifies which projects or repositories that is allowed to crawl and sets the schedule for when it should happen. 3.8 Understanding indexing schedules When you create a project or add and save a new source code location, immediately performs an initial crawl and index of the project. After that, will revisit the project based on the default global schedule. You can modify a project to use its own indexing schedule. The indexing schedule can be configured on a per-project basis, that is, each project can have a different indexing schedule. There are several ways to start the indexing process for a project: Immediately when the project is created. Based on a global schedule. By default, projects are revisited based on the global schedule. Tip: The No schedule setting can be used to disable indexing. Based on an individual schedule. On-demand with the Index Queue for next job action. 3.8.1 Setting the global indexing schedule The Settings Index Settings subtab (see Figure 3) controls the global schedule of how often Code Sight indexes your projects. It also controls which files should be ignored during the crawl. By default, when the indexing job completes, publishes the new index automatically. To keep your old index, clear the Auto Publish checkbox. This may be useful if you want to temporarily freeze the index in its current state. Page 33
Managing Projects Figure 3 Index Settings subtab To set the default (global) indexing schedule 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Settings tab. 4. Select the Index Settings subtab. 5. Click Global Schedule. The Global Indexing Schedule dialog box opens. 6. Specify how often you want to index your projects. Tip: The No schedule setting can be used to disable indexing. 7. Click Save. 3.8.2 Set the indexing schedule for a specific project By default, projects are revisited based on the global schedule. You can modify a project to use its own schedule. Page 34
Managing Projects To specify the indexing schedule for project 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Projects tab to see the projects already configured on this system. 4. Highlight a project to select it. 5. In the lower half of the screen, select the Setup subtab. 6. Click Edit Project. The Edit Project Setup dialog opens. 7. In the Advanced Settings Indexing Schedule section, specify how often you want to index this project. 8. Click Save. 3.8.3 Excluding a project from indexing At certain times you may want to freeze the current state of the project and disable indexing. To exclude a project from indexing 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Projects tab to see the projects already configured on this system. 4. Highlight a project to select it. Page 35
Managing Projects 5. In the lower half of the screen, select the Setup subtab. 6. Click Edit Project. The Edit Project Setup dialog opens. 7. In the Advanced Settings Indexing Schedule section, select No schedule. 8. Click Save. 3.8.4 Forcing to index a project There may be times when you want to index a project at a time other than the scheduled frequency or you may want to force to index a project that had been previously excluded from indexing. To force to index a project 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Projects tab to see the projects already configured on this system. 4. If necessary, use the Search field on the Projects tab to locate the project. 5. Select the checkbox next to the project that you want to index. 6. From the More actions menu, select Index Queue for next job. Page 36
Managing Projects 3.9 Viewing project history You can track the status of your projects with the Projects History subtab (see Figure 4). This page shows the operations that has performed (crawl, index, and publish) and any warnings that were generated. Page 37
Managing Projects Figure 4 Viewing the project history 3.10 Deleting a project When you delete a project it is no longer searchable. will fully remove the project the next time the index is updated (published). To delete a project 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Projects tab to see the projects already configured on this system. 4. If necessary, use the Search field on the Projects tab to locate the project. 5. Select the check box next to the project name. 6. From the More actions menu, select Delete. Page 38
Managing Projects 7. In the confirmation message, click OK. Page 39
Configuration Settings 4 CODE SIGHT CONFIGURATION SETTINGS This chapter describes configuration settings for the server. The Admin Settings tab contains product configuration settings. In general, these settings only need to be configured once, before doing any other tasks on your system. 4.1 Configuring indexing settings The Admin Settings Index Settings tab (see Figure 5) controls the global schedule for how often should index your projects. You also use this tab to configure which files should ignore while indexing. Figure 5 Index settings By default, when the indexing job completes, the new index is published automatically. You can clear the Auto Publish box if you want to keep your old index. This may be useful if you want to temporarily freeze the index in its current state. 4.1.1 Creating a global indexing schedule The global indexing schedule controls how often revisits the projects in your repositories to scan for changes that require re-indexing. To set the global indexing schedule 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Settings tab. 4. Select the Index Settings subtab. 5. Click Global Schedule. Page 40
Configuration Settings 6. In the Global Indexing Schedule dialog box, specify how often you want to index this projects 7. Click Save. Tip: You can also set an individual schedule by creating a project schedule after the project has been created. See Set the Indexing Schedule for a Specific Project for details. 4.1.2 Configuring indexing to ignore directories and files You may have files or directories in your projects that do not need to be indexed. These might include binary files, build artifacts, and SCM control files. Use the Index Settings page to indicate which files should ignore during indexing. By default, is configured to ignore certain binary file types during indexing. You can modify the list to include more or fewer entries. Note that ignoring tar or zip files here is not related to using the Zip repository type. In that case, correctly looks inside the archive to find and index your non-binary files. The following file types are ignored by default: bmp doc gzip lib xls chm exe jar pdf zip com gif jpeg png dll gz jpg tar 4.2 Configuring SCM connection timeout The Admin Settings SCM Settings tab (see Figure 2) lists the SCM clients installed on your system and detected by. If cannot access your SCM client software, then it will not show that repository type when you are adding the source location to your project. The File System, Git, and Zip clients are built into and are always present. Tip: You can set the timeout variables to prevent long delays when there are problems connecting to remote repositories. Page 41
Configuration Settings 4.3 Configuring email settings You use the Admin Settings Mail Settings tab (see Figure 6) to define how should send email to the system administrator. Currently, this is only used after a failure of an automatic upgrade. Figure 6 Mail settings Administrator Address: E-mail is sent to this address if something fails during an update. SMTP Server: Used for sending mail from this client. SMTP Port: Connection port used for sending mail. SMTP Sender: E-mail sent from comes from this address. SMTP Username and Password: Used to connect to your mail server. 4.4 Configuring proxy settings You use the Admin Settings Proxy Settings tab (see Figure 7) to specify how should access the servers at Black Duck Software. This connection information is needed for license verification and for downloading updates. Figure 7 Proxy settings 4.5 product registration There are three registration options for using : Free Unregistered: You are limited to indexing 200,000 lines of code. You can obtain a free Page 42
Configuration Settings registration key by visiting http://www.blackducksoftware.com/code-sight/upgrade. Free Registered: See the limits set forth at: http://www.blackducksoftware.com/go/codesight_additional_terms. Paid Registration: You are allowed to index projects up to a limit specified in your license agreement. The two free options display a message on the Progress dashboard, indicating that indexing is limited. Otherwise, the software is the same for all options. Note that this message is not shown to users of the search page. Typically you register immediately after installation. However, it is not required. If you do not register, the software behaves normally, except that it stops publishing after reaching 200,000 lines of code. Also, system updates are disabled until you register. Tip: At any time, unregistered users can obtain a free trial license for more lines of code by visiting the Black Duck website: http://www.blackducksoftware.com/code-sight/upgrade 4.5.1 Activating To activate your license 1. Have your enterprise or free license key ready. 2. Log in to as a user with administrator privileges. 3. Select the Admin link in the upper right corner. 4. Select the Settings tab. 5. Select the Product Registration subtab. 6. Click Activate. Page 43
Configuration Settings 7. On the registration page, enter your registration ID and the connection details specifying how your installation of will contact Black Duck for updates. 8. Click Register. Note: Your license may include an expiration date. After that time has passed, you can continue to search your projects, but further publishing is disabled. 4.5.2 Reactivating If you want to change your registration at a later time, go to Admin Settings Product Registration (see Figure 8). Enter your new registration ID and click Reactivate. Page 44
Configuration Settings Figure 8 Product registration Note: Your license may include an expiration date. After that time has passed, you can continue to search your projects, but further publishing is disabled. 4.6 Configuring updates Note: You cannot update your software until you have registered either a free or enterprise license. To perform a update 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Settings tab. 4. Select the Update Status tab. displays a page where you can select update times and Page 45
Configuration Settings a list of the most recent updates available. 5. From the Auto Install Software Update menu, select when you want new updates to be downloaded and installed: Never: You will never get updates. During Scheduled Times: You will get updates only during the times that you configure in the Day Settings section. When Available: You will get updates when they are available. 6. In the Day Settings section, specify when your server can look for updates from Black Duck. If you specify a long duration, the Black Duck site is checked repeatedly over that time interval. If you have selected the option to automatically install the update, the system handles this appropriately. 7. If you want to download and install manually, and if there are updates available, select the Download link for the first update to download it to your site. After downloading is complete, the state changes to downloaded and the action changes to an Install link. 8. Select the Install link to integrate this update into your system. 9. When installation is complete, a Delete link displays that lets you clean up the space used for download. This also removes the entry from the Active Updates list, in order to keep the list short. Page 46
Configuration Settings Note: When you connect to the Black Duck server for updates, some statistical and identification data is transmitted along with your request. See Appendix A Data Gathered by for a description of this data. Page 47
Managing the Server 5 MANAGING THE CODE SIGHT SERVER This section describes how to manage the server on an on-going basis after it is initially configured. 5.1 Disk space requirements and usage Black Duck recommends a minimum of 100GB of free disk space for the server. The installation checks for a minimum of at least 20GB, but you can quickly use up this space once you start indexing new projects. Tip: As accesses repositories and downloads new project files, it copies those files to the local system under the Black Duck user account. On Linux systems, this would be in the /home/blackduck directory. For Windows, it is under the Users area for the blackduck user. The files are then processed and indexed, with some data saved into the PostgreSQL database. During the installation, you can specify a different location for this database. Using different devices can help to manage your overall disk usage. 5.2 Startup, configuration, and log Files The following files are listed for a Linux system, when using the installation default locations. Type Server installation logs Tomcat server logs (Older version logs) (Older version logs) Startup configuration and memory management Black Duck update download and installation logs Path /tmp/black_duck_code_sight_install_<date>.log /tmp/installer_debug.txt /tmp/codesight_installlog.log /tmp/bds_iaca_codesight_server.log /opt/blackduck/codesight/tomcat/logs/catalina.out /opt/blackduck/codesight/tomcat/logs/blackduck_log.txt.<date> /opt/blackduck/codesight/tomcat/logs/blackduck_access_log. <date>.txt /opt/blackduck/codesight/tomcat/config /opt/blackduck/codesight/downloads/codesight/<version>/ <update_dir>/.download.log /opt/blackduck/codesight/downloads/codesight/<version>/ <update_dir>/.install.log The update_dir corresponds to the individual directory created for each update. The version corresponds to the current stream version of Code Sight. Page 48
Managing the Server The locations are slightly different for Windows. On Windows systems, the uses the %TEMP% environment variable to find the temp area. Depending on your version of Windows, the temp area may be in C:\Documents and Settings\<username>\Local Settings\Temp. On Windows the default installation path is C:\Program Files\Black Duck Software\CodeSight 5.3 Stopping and starting the Tomcat server running on Linux There are instances where you may need to stop and restart the Tomcat server. The following example outlines this process. It requires the following actions: Stopping the job server Stopping the Tomcat server Starting the Tomcat server Starting the job server Note: Both stopping and starting the servers require that you log in as root on Linux. To stop the Tomcat server running on Linux 1. In, go to Admin Process and click Stop. The server status changes from Running to Shutting Down to Disabled. Important: Wait for the status to change to Disabled. Stopping Tomcat without allowing all jobs to complete can lead to data consistency issues. 2. In Linux, log in as root. 3. Enter the following command: root@linux:~> /etc/init.d/bds-codesight-tomcat stop 4. Ensure that all Tomcat processes have stopped by using the following command to check the process status: root@linux:~> ps aux grep tomcat Page 49
Managing the Server 5. If the shell indicates that Tomcat is still running, enter the following command to delete the Tomcat lock: root@linux:~> rm -f /var/lock/subsys/bds-codesight-tomcat To start the Tomcat server running on Linux 6. In Linux, log in as root. 7. Enter the following start command: root@linux:~ /etc/init.d/bds-codesight-tomcat start 8. In, go to Admin Process and click Start. 5.4 Stopping and starting the Tomcat server running on Windows There are instances where you may need to stop and restart the Tomcat server. The following example outlines this process. It requires the following actions: Stopping the job server Stopping the Tomcat server Starting the Tomcat server Starting the job server To stop the Tomcat server running on Windows 1. In, go to Admin Process and click Stop. The status changes from Running to Shutting Down to Disabled. Important: Wait for the status to change to Disabled. Stopping Tomcat without allowing all jobs to complete can lead to data consistency issues. 2. In Windows, go to Start Control Panel System and Security Administrative Tools Services to open the Services Manager. Page 50
Managing the Server 3. Right-click the Black Duck Tomcat service and select Stop. To start the Tomcat server running on Windows 1. Go to Start Control Panel System and Security Administrative Tools Services to open the Services Manager. 2. Right-click the Black Duck Tomcat service and select Start. Page 51
Managing the Server 5.5 Configuring password strength By default, does not enforce any restrictions on user names or passwords. However, you can configure to enforce the following password requirements: Important: Enabling any of these password restrictions with also automatically activate the rule that prevents user names and passwords with the same value. The test for matching user ID and passwords is case insensitive. So for example, JohnDoe\johndoe would fail the validation and generate an error message. User name must contain at least x number of characters (length specified by the administrator) Password must contain at least x number of characters (length specified by the administrator) Password must contain at least one uppercase letter Password must contain at least one number Password must contain at least one special character (list of required characters specified by the administrator, for example, @ # $ % *, etc.) Password may not contain specific characters (list of illegal characters specified by the administrator, similar to required characters) Page 52
Managing the Server Step 1: Determine your password requirements Determine which password requirements you are going to enforce and what the values will be. Configuration Setting validation.user.name.minlength validation.user.password.illegalchars Values The minimum number of characters that you want to require for user names. List of characters that are illegal as part of a password, for example,!@#$%^&. Note: uses braces { } around the list of illegal characters in password error messages. validation.user.password.minlength validation.user.password.numeric validation.user.password.requiredchars The minimum number of characters that you want to require for passwords. Require a number as part of the password? (true, false) List of characters, at least one of which must be included in the password, for example,!@#$%^& Note: uses braces { } around the list of required characters in password error messages. validation.user.password.uppercase Require an uppercase letter as part of the password? (true, false) Note: Password restrictions only affect passwords created after the restrictions are configured. Passwords created before the password restrictions were configured will not be affected. In order to have all user passwords meet the new password restrictions, you must have existing users create new passwords. Step 2: Modify the Tomcat configuration settings To configure password requirements on the Tomcat server running on Linux 1. Edit the startup file /opt/blackduck/codesight/config/bds-codesight-tomcat.start to include the password requirements that you want to use. For example: -Dvalidation.user.password.uppercase=true -Dvalidation.user.password.numeric=true -Dvalidation.user.password.requiredchars=!@#$%^& -Dvalidation.user.password.illegalchars=* -Dvalidation.user.password.minlength=10 -Dvalidation.user.name.minlength=5 Page 53
Managing the Server 2. In, go to Admin Progress and click Stop. The status changes from Running to Shutting Down to Disabled. Important: Wait for the status to change to Disabled. Stopping Tomcat without allowing all jobs to complete can lead to data consistency issues. 3. Restart the Tomcat server (as root), using the following command: /etc/init.d/bds-codesight-tomcat restart To configure password requirements on the Tomcat server running on Windows 1. Use the regedit command to open the Registry Editor. 2. Expand the hierarchy to locate HKEY_LOCAL_MACHINE\SOFTWARE\Apache Software Foundation\Procrun 2.0\codesighttc6\Parameters\Java. 3. Edit the Options entry, adding the password requirements that you want to use. For example: -Dvalidation.user.password.uppercase=true -Dvalidation.user.password.numeric=true -Dvalidation.user.password.requiredchars=!@#$%^& -Dvalidation.user.password.illegalchars=* -Dvalidation.user.password.minlength=10 -Dvalidation.user.name.minlength=5 Page 54
Managing the Server 4. Exit the Registry Editor. 5. In, go to Admin Progress and click Stop. The Status changes from Running to Shutting Down to Disabled. Important: Wait for the status to change to Disabled. Stopping Tomcat without allowing all jobs to complete can lead to data consistency issues. 6. Restart Tomcat using the Services Manager. 5.6 Enabling the consistency checker The database and Solr index must remain synchronized. If your system suffers an abrupt shutdown (for example, a power failure) the database and index could become out of sync. Code Sight includes a feature that checks for consistency between the index and the database. If the consistency checker finds problems, it corrects them and triggers the crawl/index/publish cycle. If you have enabled the consistency checker, when you start Tomcat the checker runs before starting the crawl/index/publish task loop. The consistency checker compares the project and file data in the Solr index with the published schema in the database. If it finds an inconsistency, the consistency checker removes the project or files from the index and then queues the project for reindexing. When the task loop starts the crawl/index/publish cycle, the inconsistent projects are reindexed. By default the consistency checker is disabled. Enable the consistency checker by editing a configuration setting in the Tomcat startup file. The configuration setting is: Configuration Setting blackduck.codesight.consistency.check Values 0 Disabled The consistency checker will not be run. 1 Project level check The consistency checker compares project IDs and file counts in the project and repairs any inconsistencies. 2 File level check blackduck.codesight.consistency.repair True or False The consistency checker compares project IDs, project file counts, and file/document IDs then repairs any inconsistencies. Enables repair of errors flagged by the consistency checker. To enable the consistency checker on the Tomcat server, complete the following steps: Page 55
Managing the Server To enable the consistency checker on Linux systems: 1. Edit the startup file /etc/sysconfig/bds-codesight-tomcat.start to include the Java system properties for the consistency checker. For example, -Dblackduck.codesight.consistency.check=1 -Dblackduck.codesight.consistency.repair=true 2. In, go to Admin Progress and click Stop. The Status changes from Running to Shutting Down to Disabled. 3. Restart Tomcat (as root), using the following command: /etc/init.d/bds-codesight-tomcat restart To enable the consistency checker on Windows systems: 1. Use the regedit command to open the Registry Editor. 2. Expand the hierarchy to locate HKEY_LOCAL_MACHINE\SOFTWARE\Apache Software Foundation\Procrun 2.0\codesighttc6\Parameters\Java. 3. Edit the Options entry, adding the Java system properties for the consistency checker. For example, -Dblackduck.codesight.consistency.check=1 -Dblackduck.codesight.consistency.repair=true 4. Exit the Registry Editor. Page 56
Managing the Server 5. In, go to Admin Progress and click Stop. The Status changes from Running to Shutting Down to Disabled. 6. Restart Tomcat using the Services Manager. 5.7 Backing up the database Your data should be backed up on a regular basis. The process is simple, but requires some manual steps on your part. Use the following sections as guidelines for developing your own best practices for database backups. uses a single database: bds_codesight: Back up daily. This database contains project information such as source locations, plus product information such as users and roles. In addition to the database files, you should also include the following files/directories in your normal system backups: C:\BDS-CodeSight\solr OR /var/lib/bds-codesight/solr This directory contains the actual index data. While it can be recreated automatically during the crawl/index/publish cycle, this process could take many hours depending on the number of Page 57
Managing the Server projects and total lines of code you are indexing. If you do not back it up, when you restore the database you should delete the Solr directory contents and let recreate the index. If you choose to back up the index, you must back it up at the same time as the database. The database and index must remain synchronized. C:\Program Files\Black Duck Software\CodeSight OR /opt/blackduck/codesight This is your product installation directory. C:\Documents and Settings\blackduck OR /home/blackduck/codesight As crawls the source code locations of your projects, it copies files to this user-area location (which may be different on your operating system.) As with the index, this can be recreated automatically after restoration of your database. However, if you do choose to back up the source directory, it should be backed up at the same time as the index files, to keep them synchronized. 5.7.1 Creating a database backup on Linux Back up the most important data first and label your backup media carefully. The standard process for backing up a PostgreSQL database is to first copy it to a file. This serves as your backup. You may then want to copy your backup files to some removable media such as a CD or DVD. If you are ever in a situation where you lose some data or a database gets corrupted for any reason, you can restore the database using this file. The process begins with the creation of the dump file as outlined below. Important: When backing up your database, we recommend that you log in to the Code Sight Admin area, click Stop to stop the job server, and allow any current indexing jobs to finish before starting this procedure. When indexing completes (Status = Disabled), shut down the Tomcat application server. This ensures that nobody is using and making changes during the procedure. Note: is unavailable during this procedure while Tomcat is shut down. To back up the PostgreSQL database on Linux 1. Log in as root. 2. Stop Tomcat. In a distributed system, stop it from the system where it is installed. /etc/init.d/bds-codesight-tomcat stop 3. Change to the bds_codesight user. This is the user that owns the PostgreSQL installation. su - bds-codesight Page 58
Managing the Server Your prompt changes, indicating that the new user and your current directory changes to: /var/lib/bds-codesight 4. Dump the database to a location with sufficient free space (this example uses /tmp). The pg_dump command should be on bds-codesight s command search path. If not, the command is located at: /opt/blackduck/codesight/postgresql/bin. bds-codesight~> pg_dump -Fc -f /tmp/bds_codesight.dump bds_codesight This puts the bds_codesight database into a file called bds_codesight.dump in the /tmp directory. 5. Take the bds_codesight.dump file and save it on another device. 6. Change back to the root user. su 7. Restart the Tomcat server. /etc/init.d/bds-codesight-tomcat start 5.7.2 Creating a database backup on Windows Note: This procedure requires a database password. Contact Black Duck Customer Support for the password for your database. Please use the online forum to contact Support. On Windows systems, you can use the pgadmin III tool to back up your databases. This tool is included with the installation and is available in your Start menu. Important: When backing up your database, we recommend that you log in to the Code Sight Admin area, click Stop to stop the job server, and allow any current indexing jobs to finish before starting this procedure. When indexing completes (Status = Disabled), shut down the Tomcat application server. This ensures that nobody is using and making changes during the procedure. Note: is unavailable during this procedure while Tomcat is shut down. To back up the PostgreSQL database on Windows 1. Start the pgadmin tool: Start All Programs PostgreSQL 9.2 pgadmin III 2. Right-click the Black Duck database, and select the Properties option. 3. Change the username from bds-codesight to blackduck and click OK. This is a privileged user account that you will use to access the database. Page 59
Managing the Server 4. Right-click the Black Duck server, and select the Connect option. 5. Enter the password you obtained from Black Duck Customer Support. Page 60
Managing the Server 6. Right-click the bds_codesight database and select the Backup option. 7. Enter a name for the backup file, such as bds_codesight.dump, and click OK. 8. When the process completes, click Done. Page 61
Managing the Server 5.7.3 Restoring the database on Linux The restore program lets you retrieve files you have backed up with the pg_dump command. Once you have saved any (or all) of your databases to a file, you can then restore that particular database if required. Note: If you have also backed up the index and source files, they should be restored at the same time, prior to restarting Tomcat. To restore a damaged or corrupted database on Linux, complete the following steps: 1. Login as root on the system on which the database server is installed. 2. Stop Tomcat. /etc/init.d/bds-codesight-tomcat stop 3. Change to the bds-codesight user (this is the Linux user account that owns the PostgreSQL database): su - bds-codesight 4. Delete the original database (if still present), using the following command (or env): dropdb bds_codesight 5. Re-create the database using the following command: createdb bds_codesight This creates a new database and gives output that shows CREATE DATABASE. If you have made configuration modifications to improve performance, you may need to supply various options to Page 62
Managing the Server CREATE DATABASE depending on your circumstances. 6. To upload your database with your data, use the following command: psql -c "grant all on database bds_codesight to blackduck" template1 7. To upload your database, use the following command: pg_restore -d bds_codesight bds_codesight.dump Alternately, if you have a multi-core processor, you can increase the speed of the restoration by using the j qualifier to specify the number of parallel processors to use. Do not set this value higher than the number of available processors (4 to 6 is a reasonable starting point). pg_restore -j 4 -d bds_codesight bds_codesight.dump The restore may generate multiple warning messages that can be ignored. 8. Change back to the root user. su 9. Restart the Tomcat server. /etc/init.d/bds-codesight-tomcat start Once this is done, you have successfully reverted to your old database. 5.7.4 Restoring the database on Windows The restore program lets you retrieve files you have backed up with the pg_dump command. Once you have saved any (or all) of your databases to a file, you can then restore that particular database if required. Note that if you have also backed up the index and source files, they should be restored at the same time, prior to restarting Tomcat. On a Windows system, you can use the pgadmin III tool to restore your backup files. The general steps are: drop the old database, re-create it, grant access to the bds-codesight user, and then restore the data from your backup file. 1. Start the pgadmin tool: Start All Programs PostgreSQL 9.2 pgadmin III 2. Right-click the Black Duck database, and select the Properties option. 3. Change the Username from bds-codesight to blackduck, and click OK. This is a privileged user account that you will use to access the database. Page 63
Managing the Server 4. Next, right-click the Black Duck server, and choose the Connect option. Enter the password you obtained from Black Duck Customer Support. Page 64
Managing the Server 5. Drop the database you are going to replace. 6. Click Yes to confirm that you want to drop the selected database. 7. Create a new, empty database with the same name. 8. Grant access to the bds-codesight user. 9. Restore the database using the datbase dump file. Page 65
Managing the Server Page 66
APPENDIX A Data Gathered by APPENDIX A DATA GATHERED BY CODE SIGHT Periodically, your installation interacts with servers managed by Black Duck and hosted at secure data centers. Black Duck treats this communication as confidential information as further described in your agreement with Black Duck Software, Inc. The Black Duck network contains the following components: Registration Server Used to register and maintain all registration information. Update Server Provides software updates to customers when available. The following data is collected automatically to verify compliance with the Black Duck agreement: Collection date Time when data was gathered. Collection period Time since previous data collection. Update level Current system update level. Timestamp Current date and time. Update ID Unique identifier of the update being installed. Registration ID Unique registration key of your installation. System ID A random number assigned by during installation. Product name " " Product stream Version of installed. Indexed Lines of Code Total number of lines indexed for search. Indexed File Size Total size in bytes of indexed files. The following generic information is collected for product planning purposes: Java runtime environment version Operating system name CPU architecture Total system memory Free disk space in the installation directory Free disk space in the database table space directory. Page 67
APPENDIX B Using and Configuring LDAP or Active Directory APPENDIX B USING AND CONFIGURING LDAP OR ACTIVE DIRECTORY This section describes how to connect to your Lightweight Directory Access Protocol (LDAP) or Active Directory (AD) environment. This chapter assumes that you already have a server set up and working on your system. requires a valid username and password to log in. This user-based system is important for distinguishing who is able to accomplish various tasks within. If you do not have an LDAP or AD server, authenticates users against its internally-stored passwords. If you do have an LDAP server available, can authenticate against the LDAP/AD server instead. There is no difference to the user, except that they only need to maintain one password instead of two. Tip: To configure for LDAP, it may help to use an LDAP browser such as JXplorer to examine your current LDAP server settings. See http://www.jxplorer.org/ for download information. B.1 Configuring to use LDAP or AD To configure to authenticate against LDAP or AD 1. Log in to as a user with administrator privileges. 2. Select the Admin link in the upper right corner. 3. Select the Users tab. 4. Select the LDAP/AD tab. Page 68
APPENDIX B Using and Configuring LDAP or Active Directory 5. Select the LDAP is enabled check box. 6. Enter the appropriate details for connecting to your LDAP/AD server (see Table 3). You may need the assistance of your LDAP/AD manager to know how to complete the fields. 7. Click Save. You do not need to restart. Any log-in attempts made after the change will automatically use the new method. Table 3 LDAP Configuration Fields Field Description LDAP Server LDAP Authentication Type Specify the address of the LDAP/AD server you want to use for authentication. Select from the following types of LDAP authentication mechanisms: none (anonymous) Supply the Manager DN and leave the Manager password empty. simple (with a plain-text password in the request/response challenge) Supply the Manager DN and the Manager password. digest-md5 (hashed password in request/response challenge) Supply the Manager DN as a samaccountname value and the Manager password. Page 69
APPENDIX B Using and Configuring LDAP or Active Directory Field Description LDAP Manager DN LDAP Manager Password LDAP URL for Reset Password LDAP User DN Pattern If your system requires an LDAP/AD manager account for access, specify the Distinguished Name of the LDAP/AD manager account. Code Sight accesses the server using this username. Note that some systems allow anonymous access, so this field may not be required. If required, specify the password for the LDAP/AD manager account. Specify the URL a user can access to reset their password. A link to this address will appear in the user s profile page. Leave this field blank if you do not want users to be able to change their LDAP/AD passwords from within. The relative Distinguished Name (DN) pattern that the search base it appended to. Users DNs are constructed by replacing the placeholder {0} with the username supplied by the user during authentication. An example pattern would be "uid={0}, ou=developers" where a search base of "dc=mycompany,dc=com" would be appended to form an absolute DN. LDAP User Search Base LDAP User Search Filter LDAP Group Search Base LDAP Group Search Filter LDAP Group Name Attribute Local User accesses the LDAP/AD system and searches the directory information tree (DIT) for the user s login information. This field contains the starting point on the tree to use for the search. Specify an optional filter to use when searching for names to validate against the LDAP/AD server. can grant overall roles to all members of an LDAP/AD group. This field contains the starting point on the tree to use for the search. Specify the search filter that provides the list of LDAP/AD groups for the display page. Specify the attribute used to store group names in the LDAP/AD server. Specify one default user with at least the System Administrator role who will always have an account in the system. This will be the only user who can still log in using their password if your LDAP/AD system is not available. Tip: Use the default system administrator account that was set up when you first installed. Page 70
APPENDIX B Using and Configuring LDAP or Active Directory Field Description Create Users On Login Group Assigned to Users Created On Login Paging Limit Select this check box to allow new users to automatically create a new account for themselves based on their LDAP account. Otherwise, you can create user accounts in bulk by synchronizing with an LDAP system, or individually using the Create User button. Specify a default group for new users created using the "on login" process. Set this field to a value at or below the paging limit on your LDAP server. If paging is not configured on your server, you can leave this field blank. B.2 Importing and synchronizing users Your LDAP/AD server likely contains a large list of users. You can import user accounts from LDAP/AD directly into. This saves you the effort of creating users one at a time, and can be used to regularly synchronize the users with changes in LDAP. To import and synchronize users from LDAP/AD into 1. Go to the Admin Users LDAP Synchronization page. 2. Set the All Users Search Filter to the group from which the users are read. 3. Use the User Field Mapping section to indicate the LDAP element for each field that you Page 71
APPENDIX B Using and Configuring LDAP or Active Directory want filled in. In this example, only the username is transferred from the LDAP system. 4. Click Save. If the synchronization process finds accounts in that are not in LDAP/AD, it disables them. If it finds accounts in LDAP/AD that are not in, it adds them to and enables them. Accounts that exist in both systems are synchronized to match the LDAP/AD data and are enabled. Note: This process does not grant any roles to the users. It simply allows them to log in and read the online Help. You can grant overall roles to groups of users to give them additional permissions in. B.3 Granting roles to groups of users If your LDAP/AD server has users in various groups, you can choose a particular group and grant all members the same role in. 1. Go to Admin Users User Groups. The table shows the groups configured on your system. Some of these correspond to groups in your LDAP system. 2. Click Create LDAP Group. connects to your LDAP server and shows a list of your LDAP groups. 3. Select an entry from the table of LDAP Groups. Page 72
APPENDIX B Using and Configuring LDAP or Active Directory 4. Click Save. This creates a group with the same name as in LDAP. The new group in is empty. LDAP users in the group are automatically added to this new group when they log in. 5. Click Add Overall Roles to add a system-wide role for all members of the group. 6. Click Add Project Role to add a project-specific role for all members of the group. Page 73
APPENDIX B Using and Configuring LDAP or Active Directory B.4 LDAP configuration examples The following sections contain LDAP configuration examples. B.4.1 Authentication searches directly below a domain component node It is possible to configure LDAP to search directly below a domain node. For example, if the domain is specified as blackducksoftware.com, then it is possible to search for users directly below the blackducksoftware domain node as well as recursively within any child nodes. The Jxplorer screenshot shown in Figure 9 shows an example LDAP node tree with domain nodes com and blackducksoftware. Domain nodes are indicated with a blue dot as well as having an objectclass attribute of domain listed in the right-hand side attributes table. Figure 9 Jxplorer view of a Domain component In this example, the goal is to find users directly below the domain node blackducksoftware (for example, users Bob Brady and Olga Garber) as well as users in the organizational units (indicated with the node tree icon) such as Development, IT, Sales, and any others matching a filter query. The LDAP configuration is shown in Figure 10. The fields that are key to the search are, LDAP User Search Base, LDAP User DN Pattern, and LDAP User Search Filter. The search base specifies the starting node from where recursive searches will take place. Because the goal is to start the search from directly below the blackducksoftware domain node, the value given is dc=blackducksoftware,dc=com. Note the order of the node elements in the search base: it starts with the child node and goes up the tree to parent nodes when read left-to-right. This ordering is critical because it is an LDAP convention that is used with specified search criteria and distinguished names (dn values). The user dn pattern is relative to the search base. The {0} in the field will be replaced by the username supplied by the user during login. The LDAP software will construct the search dn by taking the user dn Page 74
APPENDIX B Using and Configuring LDAP or Active Directory pattern and prepending it to search base. In the configuration example, the constructed dn will be samaccountname={0},dc=blackducksoftware,dc=com. Jxplorer can be used to verify that the format of the constructed dn is appropriate. Note: This example uses screen captures from Black Duck Code Center, but the principles are the same for. Figure 10 LDAP configuration for searches directly below a domain component The Jxplorer screenshot shown in Figure 11 shows the attributes, including dn, for user node Bob Brady which is a direct child node of the domain node blackducksoftware. Note that the samaccountname is bbrady. The user should use bbrady as the username during login. Furthermore, the samaccountname=bbrady,dc=blackducksoftware,dc=com dn is equivalent search-wise to the listed dn cn=bob Brady,dc=blackducksoftware,dc=com, so the constructed search dn in the configuration should work successfully. The entire tree below the blackducksoftware node will be searched for users having a dn that matches the constructed one. If the user attempting to authenticate cannot be found with the given dn pattern, then the search filter will be used in an attempt to find the user. The filter samaccountname={0} given in the configuration will be used by the search if the user cannot be found with the constructed dn pattern. Since the example LDAP tree is configured so that every user has a samaccountname attribute, users like Page 75
APPENDIX B Using and Configuring LDAP or Active Directory Ronan Fagan (see Figure 12) in ou Sales will still be found. Figure 11 Jxplorer view of a user directly below a domain component Figure 12 Jxplorer view of a user in an organizational unit below a domain component Page 76
APPENDIX B Using and Configuring LDAP or Active Directory B.4.2 Authentication searches restricted to a branch node It is also possible to configure the LDAP authentication searches so that they are restricted to a branch within the LDAP tree. For example, if you need to restrict access to only users within the ou Development within the LDAP tree shown in Figure 11, then the search base in the Figure 12 configuration would be changed to be ou=development,dc=blackducksoftware,dc=com, the dn pattern and search filter could be left unchanged as samaccountname={0}. Note: The ordering within the search base text is essential. The lowest level child node, Development, is given first, with each successive parent node from the above level added as the text is read from left to right. The search base restricts searches to within the development.blackducksoftware.com branch, even if the dn pattern search should fail and the filter be invoked. This means that users located outside the branch, like Bob Brady in the previous example, would be unable to successfully authenticate, even if they passed a valid password at login. Page 77