Simba ODBC Driver with SQL Connector for Apache Spark

Transcription

1 Simba ODBC Driver with SQL Connector for Apache Spark for Windows Simba Technologies Inc. Revised: January 9, 2015

2 Contents Purpose... 3 Setting Up Hadoop... 3 Setting Up Apache Spark... 3 Importing Sample Data into Spark... 4 Installing the Simba ODBC Driver with SQL Connector for Apache Spark... 6 Configuring the Simba ODBC Driver with SQL Connector for Apache Spark... 6 Connecting to the Data Source... 7 Using Excel... 7 Using Tableau... 8 Contact Us Appendix: Troubleshooting Architecture Mismatch Problems For assistance at any point in this installation process, please contact Simba for free Engineering Level Support at: techsupport@simba.com. Page 2

3 Purpose The Simba ODBC Driver with SQL Connector for Apache Spark for Windows is targeted towards users of the Simba ODBC Driver with SQL Connector for Apache Spark, showing you how to set up your environment quickly so that you can evaluate and use the driver. Setting Up Hadoop To use Spark, you need to have a Hadoop installation already set up. You can download Hadoop from Hadoop installation guides are available at Alternatively, you can download and use one of the following pre-configured Hadoop distributions: Hortonworks: Cloudera: MapR: +Test+Drive+MapR+on+a+Virtual+Machine Setting Up Apache Spark You can download Spark from Spark installation guides are available at To confirm that Apache Spark is installed and running: 1. Start Hadoop, then open the command line interface on the machine that Hadoop is running on. 2. To see a list of the Hadoop processes currently running, type the following command and then press ENTER: jps The command returns a list that looks similar to the following: 2886 TaskTracker 3069 JobTracker 2783 DataNode RunJar 2670 NameNode Jps Page 3

4 Note: If your Hadoop distribution is configured to use secondary name nodes, then you may also see a SecondaryNameNode item in the list. 3. To confirm that Spark is running, type the following command, where PortNumber is the port on which Spark is listening: netstat nl grep PortNumber Note: The default port number for Spark is If Spark is running correctly and listening on an open port, then a line of text similar to the following appears in the Command Prompt window, where PortNumber is the port on which Spark is listening: tcp :PortNumber :* LISTEN Importing Sample Data into Spark The following procedure describes how to import some sample data if you do not already have data to use. The Hadoop and Spark configuration shown in the procedure is on Linux; however, you can use the same steps for a configuration that is hosted on Windows. To import sample data into Spark: 1. Download the sample data set available at Note: This is a modified version of the FAA data set. The original FAA data set is available for download at 2. Create a new folder in a convenient location, and then extract the sample data set into the folder. Note: The SparkSchema.txt file contains the Spark CREATE TABLE commands for each of the CSV files in the data set. Use these commands if you want to load tables other than the Airline table used in this procedure. 3. Start Hadoop and Spark as directed in the respective user guides. Documentation for Hadoop is available at Documentation for Spark is available at Page 4

5 4. Open a Command Prompt window and browse to the directory containing the binaries for Spark. By default, the binaries are located in the bin subfolder of the Spark installation. 5. To run the Spark-SQL shell, type the following command, and then press ENTER:./spark-sql If the command runs successfully, then the following prompt appears in the Command Prompt window: spark-sql> 6. In the shell, create a table to hold the data from the Airline.csv file by typing the following command and then pressing ENTER: create table Airline( UNIQUE_CARRIER string, AIRLINE_ID int, CARRIER string, TAIL_NUM string, FL_NUM string ) row format delimited fields terminated by ',' stored as textfile; If the command runs successfully, then text similar to the following appears in the Command Prompt window, where Seconds is the number of seconds it took to run the command: OK Time taken: Seconds 7. To import data from the Airline.csv file included in the sample data that you downloaded in step 1, type the following command, where Path is the full path to the Airline.csv file: load data local inpath 'Path/Airline.csv' into table Airline; If the command runs successfully, then text similar to the following appears in the Command Prompt window, where Path is the full path to the Airline.csv file and Seconds is the number of seconds it took to run the command: Copying data from file:path/airline.csv Copying file: file:/path/airline.csv Loading data to table default.airline OK Time taken: Seconds 8. Confirm that the data has been imported into Spark by running the following command: select * from Airline; Page 5

6 If the data was imported successfully, then output similar to what is shown in Figure 1 appears in the Command Prompt. Figure 1 Contents of the Sample FAA Collection Named Airline Installing the Simba ODBC Driver with SQL Connector for Apache Spark The Simba ODBC Driver with SQL Connector for Apache Spark is available for download from For details on installing the driver, refer to the Simba ODBC Driver with SQL Connector for Apache Spark Installation and Configuration Guide, which is available for download on the Documentation tab at Configuring the Simba ODBC Driver with SQL Connector for Apache Spark To use the Simba ODBC Driver with SQL Connector for Apache Spark, you need to configure a Data Source Name (DSN) to connect to your data source using a client application, such as Microsoft Excel or Tableau. Page 6

7 To create a Data Source Name: 1. Click the Start button, then click All Programs, then click the Simba Spark ODBC Driver 1.0 program group corresponding to the bitness of the client application accessing Spark, and then click ODBC Administrator 2. Click the System DSN tab. 3. Select the Sample Simba Spark DSN in the list of DSNs, and then click Configure 4. Configure your DSN by doing the following: a. In the Host field, type the IP or hostname of your Spark server. b. In the Port field, type the number of the port that Spark is running on. Note: The default port number for Spark is c. In the Database field, type the name of the database that has your data in it. To access the airline table that was created earlier, type faa d. Optionally, if your Spark server is configured to use authentication, then select the appropriate authentication mechanism from the Mechanism list and provide the necessary credentials and authentication settings in the fields below the list. 5. To test your connection to Spark, ensure that the Spark server is running, and then click the Test button in the Simba Spark ODBC Driver DSN Setup dialog box. The Test Results dialog box appears, displaying a message informing you if the connection succeeded or failed. Click OK to close the Test Results dialog box. 6. Click OK to close the Simba Spark ODBC Driver DSN Setup dialog box, and then click OK to close the ODBC Data Source Administrator. Connecting to the Data Source After creating a Data Source Name for connecting to your Spark data source, you can connect to the data source using a client application, such as Microsoft Excel or Tableau. Using Excel Note: The following procedure is written for Microsoft Excel To connect to your Spark data source using Excel: 1. In Excel, select the Data tab in the ribbon, then click From Other Sources in the Get External Data group, and then click From Data Connection Wizard Page 7

8 2. In the Data Connection Wizard, select ODBC DSN in the list of data source types, and then click Next 3. In the list of ODBC data sources, select Sample Simba Spark DSN (or another DSN that you have created and tested) and press Next If the Data Link Properties dialog box appears, then the connection to the driver did not succeed. If you press the Test Connection button in the Data Link Properties dialog box, the following error appears: Figure 2 Data Link error message Important: If the connection fails, confirm that the bitness of the driver that you are using matches the bitness of Excel. If you are using the 32-bit version of Excel, then you need to use the 32-bit version of the driver. If you are using the 64-bit version of Excel, then you need to use the 64-bit version of the driver. For details about diagnosing the issue, see Architecture Mismatch Problems on page In the list of tables, select airline, and then click Finish 5. In the Import Data dialog box, you can specify the cell where you want the top-left corner of the airline table to appear. Leave the location set to the default value (=$A$1) and click OK Data from the retrieved airline table appears in the current Excel worksheet. Using Tableau Note: The following procedures are written for Tableau 8. The Simba ODBC Driver with SQL Connector for Apache Spark also supports Tableau 7. The Simba ODBC Driver with SQL Connector for Apache Spark includes a Tableau Data-connection Customization (TDC) file for Windows that helps configure Tableau to operate more efficiently using the driver. Page 8

9 To install the.tdc file: 1. Copy the file named Simba Spark ODBC Driver.tdc from the driver installation directory to the Documents\My Tableau Repository\DataSources directory. 2. Restart Tableau. To connect to your Spark data source using Tableau: 1. In Tableau, click Connect to Data in the left navigation pane, and then scroll down to select Other Databases (ODBC) 2. In the DSN list, select Sample Simba Spark DSN and then click Connect Important: If the connection fails, confirm that the bitness of the driver that you are using matches the bitness of Tableau. If you are using the 32-bit version of Tableau, then you need to use the 32-bit version of the driver. If you are using the 64-bit version of Tableau, then you need to use the 64-bit version of the driver. For more information about diagnosing the issue, see Architecture Mismatch Problems on page Select the Single Table option and click the magnifying glass to open a list of tables. 4. Select the airline table and then click Select Note: If you see the Load link, click it to see all the tables that you can select from. 5. To load the selected table, click OK If the following warning appears, press OK: This warning appears because Tableau sees only a generic driver and attempts to determine the capabilities of the driver. The warning does not affect how the driver will operate in Tableau. 6. In the Data Connection dialog box, click Connect live so that Tableau does not import all of the data and you work on the data in Hadoop. Page 9

10 Data retrieved from the airline table appears in Tableau, with the columns listed as dimensions or measures depending on the data type. You can now create reports on the table you selected. Contact Us If you have difficulty using the driver, please contact our Technical Support staff. We welcome your questions, comments, and feature requests. Technical Support is available Monday to Friday from 8 a.m. to 5 p.m. Pacific Time. Important: To help us assist you, prior to contacting Technical Support please prepare a detailed summary of the client and server environment including operating system version, patch level, and configuration. You can contact Technical Support via: support@simba.com Web site: Telephone: (604) Extension 3 Fax: (604) You can also follow us on Page 10

11 Appendix: Troubleshooting Architecture Mismatch Problems If you encounter an error message similar to The specified DSN contains an architecture mismatch between the Driver and Application, then the bitness of the application that you are using to connect to the Spark data source does not match the bitness of the Simba ODBC Driver with SQL Connector for Apache Spark. You may be attempting to connect a 32-bit application to a 64-bit driver, or vice versa. Ensure that the bitness of your application matches the bitness of driver that you are trying to use. Note: All versions of Excel 2007 and earlier are 32-bit applications. To determine the bitness of Excel 2010: In Excel 2010, click the File tab in the ribbon, and then click Help in the left navigation pane. In the right pane, the first line of text in the About Microsoft Excel area displays the bitness of your Excel application next to the version number. Figure 3 highlights the location of the version number and bitness. Figure 3 Determining the bitness of Excel 2010 Page 11

12 To determine the bitness of Excel 2013: In Excel 2013, click the File tab in the ribbon, then click Account in the left navigation pane, and then click the About Excel button. In the About Microsoft Excel dialog box, the first line of text displays the bitness of your Excel application next to the version number. Figure 4 highlights the location of the version number and bitness. To determine the bitness of Tableau: Figure 4 Determining the bitness of Excel 2013 In Tableau, click the Help menu, and then click About Tableau The About Tableau dialog box displays the bitness of your Tableau application in the top right corner, next to the version number. Page 12