Developing Business Intelligence and Data Visualization Applications with Web Maps

Developing Business Intelligence and Data Visualization Applications with Web Maps

Introduction Business Intelligence (BI) means different things to different organizations and users. BI often refers to complex business applications, tools, and technologies focused at understanding business metrics and performance. This includes such concepts as data warehouses, data cubes, dimensional analysis (where time is often one of the dimensions), and performance monitoring. BI often involves users accessing this data in spreadsheets, pivot tables, and reports. BI typically represents systems where business data is aggregated from different enterprise systems with the goal of uncovering trends and insights. These solutions are typically costly, taking years to build, and are targeted at specific business users such as analysts or CFOs. Data often propagates slowly through these systems before useful analysis can be performed. Often analysis insights come too late to effect short term business trends. Part of the issue is the complexity of BI tools that too often require sophisticated understanding and specialized skills. Portals have also begun to appear in many BI scenarios, where business data can be visualized in score cards or dashboards providing quick access for targeted use i.e. VP of Sales can log into their intranet portal site and see sales trends and details. The term Data Visualization is often used in these scenarios and applications are targeted at providing better visualization of business data. Some Geospatial Information System (GIS) vendors have been offering solutions and services in the BI space, in which data can be analyzed geographically. However this does not typically address the issues of cost, complexity, and ease of use. Most organizational datasets have a location component, such as assets, competitors, sales territories etc. And whether this is in a GIS or other system, the ability to see business data with a location context, layered with other relevant data provides real insight. Many organizations have invested heavily in BI applications, portals, and GIS software for their users. However, more often than not, these applications are point solutions i.e. solutions implemented specifically for a short term departmental need and not part of a larger strategy. Organizations should look at BI features as a set of capabilities that can be leveraged over and over again in different parts of the organization. Microsoft offers various products that enable business intelligence and data visualization solutions. This includes capabilities for creating business data and data aggregates, storage of spatial and non-spatial data, processing and querying of business data, with reporting and data visualization. Microsoft s Business Intelligence strategy is all about enabling better business insights for all users, not just specialized BI, GIS, or management groups. Web mapping platforms like Bing Maps also add specific value to Business Intelligence solutions, providing intuitive and cost effective data visualization capabilities. This can be enabled by using familiar reports (with embedded maps), tools (e.g. Spreadsheets), maps, and portals to allow any users to access business insights in an intuitive manner. 3

Figure 1- Shows a SharePoint portal application showing sales trends and related retail locations. Figure 2 Shows Bing Maps Silverlight with emergency response data and analysis overlays including incidents, 1-minute drive times and other points of interest. 4

Solution Overview and Architecture Business intelligence and data visualization solutions often involve many components, hence the traditional complexity and cost. With newer Microsoft based tools, many of the elements of BI and data visualization solutions can be implemented with reduced cost and complexity. One approach to understand BI systems architecture is the Microsoft Single View Platform. This provides guidance on how to create integrated data visualization solution leverage many organizational datasets. Web mapping technologies like Bing Maps can complement solutions by enhancing data visualization components. Figure 3 Shows data visualization components in the Microsoft Data Visualization Platform. Organizational data can come from many different operational systems; these can include sales information from retail Point of Sale (POS) systems, production data from manufacturing, field and asset data from fleet management and asset systems, customer service data from CRM systems, outage data from service and support systems, and the list goes on. This data can reside in different systems involving different technologies, databases, and data structures. This varied source data for many BI solutions is typically in the form of transactional data e.g. sales orders, shipments, etc. If your organization has not yet gone down the BI road, then don t worry. You can augment existing systems or reports with web mapping capabilities like Bing Maps. For example, using SQL Server Reporting Services R2, you can easily integrate map views with existing sales reports for a particular time period. These new reporting features allow for easy inclusion of Bing Maps with data in reports. Figure 4 Shows the SQL Server 2008 R2 Report Builder allowing maps options to be configured for reports, including the use of Bing Maps. 5

In addition to enterprise data sources, organizations can also access external or third party datasets to enhance business insights. These could include, for example, weather data or projected hurricane paths adding intelligence for weather events affecting operations. Data Warehousing is the process of aggregating data from different enterprise systems (and external data sources). The first step is to create the processes to access and load data from different systems to a central standardized format. This is also known as an ETL (extract, transform & load) process. If you don t have this process in place today, SQL Server 2008 Integration Services (SSIS) provides many of these features. Data is typically denormalized during this process (often in a star schema) to minimize lookup data for faster queries. The next step is often to pre-calculate possible results. This is known as creating a data cube via Online analytical processing (OLAP) where for example the sales statistics for days, weeks and months are pre-calculated results. Hence when it comes time to do BI analysis all the possible combinations exist without having to perform complicated joined queries on the fly. This lets users quickly explore data and trends. If you don t have these processes in place today, SQL Server 2008 Analysis Services (SSAS) provides these capabilities. User tools for BI and data visualization as discussed previously are typically very complicated with specialized analysis and/or GIS tools. More recently, however, many users have been accessing BI data using spreadsheets like Microsoft Excel. Typically this data is in a tabular format with pivot tables and using Visual Studio Tools for Office (VSTO) with Excel for example, task panes can be added next to the spreadsheet allowing data to be more easily visualized in the content of the Excel environment. Web mapping capabilities like Bing Maps can easily be embedded alongside the spreadsheet. See Figure 5 and 6 for more information. Figure 5 Show an Excel sales spreadsheet with sales person locations shown on Bing Maps in a VSTO task pane. 6

Figure 6 Shows a drill down from sale person location in Bing Maps to access more information and trends. Reporting tools and reports are common in most organizations. Using web maps capabilities like Bing Maps you can enhance existing reports or create new reports with a compelling data visualization features. Per previous discussions, you can use SSRS R2 to easily incorporate Bing Maps into reports. You can also include Bing Maps with any reporting product. In addition to the SQL Server 2008 BI and reporting features, SQL Server 2008 includes new spatial data types and spatial functions, which enable organizations to easily add location capabilities to existing data such as customer locations, retail locations etc, stored as geospatial data. Using SQL Server 2008 spatial functions, spatial queries can be created to access and analyze business data with a geographical context. These spatial data capabilities combined with existing business data and business intelligence features can provide significant business insight. Portals are a common way to allow business users to access BI information and perform data visualization. Portals offer the framework for enterprise user authentication, access, and collaboration so data can be easily presented to users of all types, customized based on their membership roles. Microsoft SharePoint Portal Server 2007 provides a platform for building collaborative portal applications. This provides a platform where maps and analysis features can be integrated acting as glue between BI and mapping capabilities. Web maps like Bing Maps offer unique value in portals, where data can be easily visualized on map views from data cube/bi datasets, SharePoint lists, and/or SQL server. Additionally, data that is visualized can also be edited with changed saved back to source systems, making for a very interactive environment. There are a number of options for adding Bing Maps within a SharePoint portal application. 7

1) Using the Bing Maps AJAX or Silverlight controls, the maps can be embedded in the portal as a customer web part. In SharePoint 2010, this becomes even easier with the direct support for Silverlight applications. The Silverlight XBAP file can be directly referenced in a SharePoint web part. 2) Using third party tools, various Microsoft partners have implemented integration with SharePoint a. IDV Solutions Their Visual Fusion Server integrates with SharePoint. This requires server components to facilitate the integration. b. ESRI Their MapIT product has a native SharePoint web part that can be implemented with SharePoint installations. c. Component One Their Maps for SharePoint control can also be implemented with any SharePoint installation and easily allow integration of SQL or SharePoint list data. Overall, if you want full control over the mapping web part features and BI data integration, it is likely easier to implement a custom web part with Silverlight and.net. SharePoint Portal Server 2007 also contains other features that can facilitate BI and Data Visualization solutions Excel Services which allow spreadsheet data to be centrally stored, and hence can be geo-coded and mapped. Enterprise search, which could allow other enterprise datasets to be found and visualized. Business Data Catalog, allows connections to data sources from within SharePoint. Lists, some enterprise data such as employee contact information could be stored directly in SharePoint lists, and serve as a map layer. Figure 7 Sample BI/Data Visualization and Web Mapping Architecture 8

Bing Maps is a web mapping platform that provides significant value for business intelligence and data visualization solutions. Bing Maps services can be used to accurately pinpoint locations from addresses (a.k.a geocoding). This latitude, longitude data can be stored in enterprise systems and/or SQL 2008 spatial data types and then aggregated in BI systems. Bing Maps provides base maps and imagery; hence you can overlay your customer locations, analysis data, and any other information in the web mapping interface. This is done using the Bing Maps APIs including JavaScript/AJAX or Silverlight Controls. Data from SQL Server or other BI data sources can be easily visualized without the complexity of traditional GIS systems. Bing Maps maintains and updates road, geo-coding, and imagery datasets so organizations can focus on core business data and analysis to provide timely business insights. Figure 8- and Features Showing Bing Maps platform, datasets, APIs and applications. 9

Bing Maps APIs AJAX/JavaScript APIs These APIs are primarily for web browser development, so are a good fit for data visualization apps. There is direct support for KML, GeoRSS, native vectors, and raster tile layers. Advantages of these APIs include working directly in a user s browser without specialized plug-ins beyond the ubiquitous JavaScript. Many UI features (such as panning/zooming etc) are part of the APIs. UI development with these API building blocks is relatively easy, and entry skill level is not high. JavaScript will also work with many web server technologies making the implementation very flexible. Disadvantages are seen with the richness of possible applications, and performance limitations due to JavaScript implementations. JavaScript can be integrated with SharePoint Web Parts. SOAP/XML APIs SOAP/XML web service APIs support server side calls. Requests can be made with rich result objects in XML. Advantages include ease of use with.net and other development technologies that support SOAP/XML, running as multithreaded background services, (e.g. bulk geo-coding), and support of any type of UI, including older browser versions, mobile, and smart clients. Disadvantages are that maps returned are static, lacking any navigation controls, so these features need to be developed as part of the application. These APIs are well suited to integrating maps in reports and geocoding organizational location data etc. Silverlight APIs These APIs use Microsoft Silverlight technology, which are primarily.net based APIs with code that runs inside the user s browser. Advantages include ease of development with.net, powerful UI features such as animation, rich graphic design, and transparency. In addition managed client side code means high performance display of large vector data sets and in some cases GPU accelerated graphics (10x performance increases are not uncommon). Similar to AJAX, these APIs are well suited to data visualization either with or without SharePoint. Important Features Bing Maps AJAX Control Bing Maps Web Service Bing Maps Silverlight Control Data Visualization Yes No Yes Report Integration Limited Yes Limited Spreadsheet Integration Yes, via IE control in VSTO Yes Limited Bulk geo-coding organizational data/ locations No Yes No Portal Integration Yes, via Web Parts Limited Yes, via Web Parts in SharePoint 2010 10

Conclusion Web mapping platforms like Bing Maps offer significant value to BI and data visualization solutions. This can include integration with existing BI systems or creating new reports, spreadsheets, and portal data visualization features. You can take advantage of web maps at any level to enhance solutions and provide improved business insight. More Information Bing Maps Overview - http://www.microsoft.com/maps/ Bing Maps APIs - http://www.microsoft.com/maps/developers/ Microsoft Business Intelligence - http://www.microsoft.com/bi/ Microsoft SQL Server 2008 BI - http://www.microsoft.com/sqlserver/2008/en/us/business-intelligence.aspx Microsoft SQL Server 2008 Spatial Data - http://www.microsoft.com/sqlserver/2008/en/us/spatial-data.aspx Microsoft SQL Server 2008 R2 - http://www.microsoft.com/sqlserver/2008/en/us/r2-ssbi.aspx Microsoft Single View Platform - http://www.microsoft.com/industry/government/solutions/single_view_platform 11