GNSS (GPS) Test Guidance Abstract This white paper details global positioning system (GPS) implementation guidance for certification of Windows 8 PCs to help ensure a high-quality and competitive GPS experience in Windows 8. The guidelines in this document apply to OEMs, IHVs, and other partners (such as software vendors). The focus is testing the integration of global navigation satellite system (GNSS) devices to the system under test and Windows 8. This information applies to the following operating systems: Windows 8 Release Preview Windows Server 2012 References and resources discussed here are listed at the end of this paper. The current version of this paper is maintained on the Web at: GNSS (GPS) Test Guidance Disclaimer: This document is provided as-is. Information and views expressed in this document, including URL and other Internet website references, may change without notice. Some information relates to prereleased product which may be substantially modified before it s commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here. You bear the risk of using it. Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.
GNSS (GPS) Test Guidance - 2 Document History Date Change First publication Contents Introduction... 4 Non-Goals... 4 Test Approach...4 Terminology...4 Glossary of Terms and Abbreviations...5 Acceptance Test Matrix...5 Requirements from Partners... 7 Utility to clear the A-GPS data... 7 GPS Acceptance Test Matrix...7 Telemetry...7 Antenna Requirements from OEM... 7 Operator Certification Tests... 8 Dependency on Third-Party Services or Win32 applications... 8 USB Selective Suspend (if applicable)...8 Firmware Updates... 8 Test Descriptions...8 Functionality... 8 1. Sensor category, type, properties and data fields... 8 2. State Transitions...9 3. Accuracy of Latitude and Longitude...9 4. Minimum Report Interval...10 5. Report Interval Set... 10 6. Report Interval Delay... 10 7. Speed Data... 10 8. Heading Data...11 9. Other Sensor Properties...11 10. Radio State and Flight Mode...11 11. Basic PnP... 11 Assisted GPS...12 1. A-GPS...12 2. Position Injection...12 Robustness...13 1. Driver Verifier, WDF Verifier and Application Verifier...13 2. Telemetry Data...13 3. GPS State Transitions... 14 4. GPS Stress Tests... 14 Performance... 14 1. Cold Start TTFF... 14 2. Hot Start TTFF...15 3. Acquisition Sensitivity... 15
GNSS (GPS) Test Guidance - 3 4. Tracking Sensitivity...15 5. Reacquisition Time...15 6. Static Navigation Accuracy...16 7. Dynamic Navigation Accuracy...16 Power Consumption... 16 1. No GPS Clients...16 2. GPS Radio OFF...17 3. Radio State and Flight Mode and Power Management...17 4. USB Selective Suspend... 17 Power Management... 17 1. Connected Standby... 17 Antenna Performance...18 1. Performance Tests with OTA Connection... 18 2. Human Interference Tests...19 Interoperability...20 1. Mobile Broadband, Wi-Fi, Bluetooth, Camera Interoperability...20 Drive Tests... 21 Simulator Tests... 21 Reporting and Results Communication...22 Test Equipment... 22 Toolset... 22 Resources... 22
GNSS (GPS) Test Guidance - 4 Introduction Non-Goals This white paper details global positioning system (GPS) implementation guidance for certification of Windows 8 PCs to help ensure a high-quality and competitive GPS experience in Windows 8. The guidelines in this document apply to OEMs, IHVs, and other partners (such as software vendors). The focus is testing the integration of global navigation satellite system (GNSS) devices to the system under test and Windows 8. Testing of areas other than GPS is out of the scope of this document. Testing of the components that interact with the location platform and devices will be limited to interoperability testing. Exercising the operating system components or the GNSS device fully is out of scope of this document. It is assumed that the IHVs and OEMs will thoroughly test their GNSS device both independently and integrated into system prior to sending it to Microsoft for managed program certification. This testing should include successful completion of the WHCK tests, this test plan, pre-operator trial tests, and internal tests developed specifically for the GNSS driver and GNSS receiver. Test Approach Testing focuses on following areas: 1. Functionality 2. Assisted GPS (A-GPS) 3. Robustness 4. Performance 5. Power 6. Antenna Performance 7. Interoperability 8. Drive Testing 9. Simulator Testing Managed program partners run the tests specified in this test plan and fill in the acceptance test matrix before submitting the systems/devices/drivers or update to these to Microsoft. The failures in these tests should be documented by the partners and will be evaluated by Microsoft GPS test team whether to accept the device for testing and how to prioritize testing efforts.
GNSS (GPS) Test Guidance - 5 Terminology GPS GPS is used interchangeably with GNSS. Unless it is stated otherwise, it should be understood to refer generally to a satellite positioning as location provider solution, rather than the Global Positioning System satellite system deployed by the US government. Clear sky conditions GPS/GNSS satellites received without obstruction from above or surrounding environment down to an elevation mask of 5 degrees above the horizon. All signal levels consistent with unobstructed signal levels at the ground and not to be lower than -131 dbm. Glossary of Terms and Abbreviations GPS: Global Positioning System GNSS: Global Navigation Satellite System GLONASS: Acronym for Globalnaya navigatsionnaya sputnikovaya sistema (in Russian) A-GPS: Assisted GPS, specifically time, coarse position and long-term orbit data TTFF: Time to First Fix RF: Radio Frequency WHCK: Windows hardware Certification Kit ADK: Assessment and Deployment Kit DUT: Device under Test CS: Connected Standby WDTF: Windows Driver Test Framework SDT: Sensors Diagnostics Tool SV: Silicon Vendor IHV: Independent Hardware Vendor OEM: Original Equipment Manufacturer SoC: System on Chip CTIA: Cellular Telecommunications & Internet Association TIS: Total Isotropic Sensitivity UHIS: Upper Hemisphere Isotropic Sensitivity PIGS: Partial Isotropic GPS Sensitivity Acceptance Test Matrix Acceptance tests
GNSS (GPS) Test Guidance - 6 Acceptance tests Test Description Phase 1 (Inbox Criteria) 1 Driver must be signed with the IHV certificate 2 Driver must be installed via device manager/dism 3 Location Sensor WHCK tests: Device.Input.Sensor.* 4 Radio Management WHCK tests: System.Client.RadioManagement.* 5 Device Fundamentals WHCK tests: Device.DevFund.* 6 System Fundamentals Power Management WHCK Tests: System.Fundamentals.PowerManagement.CS.* 7 USB WHCK tests (USB connected devices only): Device.Connectivity.UsbDevices.* 8 GPS.Test Descriptions.Functionality.Sensor category, type, properties and data fields 9 GPS.Test Descriptions.Functionality.State Transitions 1 GPS.Test Descriptions.Functionality.Accuracy of 0 Latitude and Longitude 1 GPS.Test Descriptions.Functionality.Minimum Report 1 Interval 1 GPS.Test Descriptions.Functionality.Report Interval 2 Set 1 GPS.Test Descriptions.Functionality.Speed Data 3 1 GPS.Test Descriptions.Functionality.Heading Data 4 1 GPS.Test Descriptions.Functionality.Radio State and 5 Flight Mode 1 GPS.Test Descriptions.Functionality.Basic PnP 6 1 GPS.Test Descriptions.Assisted GPS.A-GPS 7 1 GPS.Test Descriptions.Assisted GPS.Position Injection. 8 Connection Type 1 GPS.Test Descriptions. Power Consumption.No GPS 9 Clients 2 GPS.Test Descriptions. Power Consumption.GPS Radio 0 Off 2 GPS.Test Descriptions. Power 1 Management.Connected Standby with Active Clients 2 GPS.Test Descriptions.Power Management.Resume in 2 No Coverage Area 2 GPS.Test Descriptions.Power Management.Connected 3 Standby WHCK Tests with Sensor Active Clients Verification Results Comment s
GNSS (GPS) Test Guidance - 7 Acceptance tests 2 GPS.Test Descriptions.Antenna Performance.OTA 4 Connection (system should get a fix in clear sky outdoors as is no external antennas or other modifications) 2 GPS.Test Descriptions.Interoperability.* (GPS can get a 5 fix when MB, Bluetooth, Wi-Fi, camera is in active use) Phase II 1 GPS.Test Descriptions.Functionality.Report Interval Delay 2 GPS.Test Descriptions.Functionality.Other Sensor Properties 3 GPS.Test Descriptions.Assisted GPS.Position Injection. Connection Time 4 GPS.Test Descriptions.Robustness.Driver Verifier, WDF Verifier and Application Verifier 5 GPS.Test Descriptions.Antenna Performance.HumanInterference Tests (system should get a fix in clear sky outdoors as is no external antennas or other modifications while being held in common handgrips) Stress 1 GPS.Test Descriptions.Robustness.* Performance 1 GPS.Test Descriptions.Performance.* Power 1 GPS.Test Descriptions.Power Consumption.* 2 GPS.Test Descriptions.Power Management.* 3 System Fundamentals Power Management WHCK Tests System.Fundamentals.PowerManagement.* Antenna Performance 1 GPS.Test Descriptions.Antenna Performance.* Drive Tests 1 GPS.Test Descriptions.Drive Tests.* Simulator Tests 1 GPS.Test Descriptions.Simulator Tests.* Requirements from Partners Utility to clear the A-GPS data IHVs shall provide a utility to clear the assistance data to enable cold start testing and avoiding false failures in simulator testing. The utility shall have a command line interface.
GNSS (GPS) Test Guidance - 8 GPS Acceptance Test Matrix Telemetry OEMs and/or IHVs shall run the tests specified in the GPS Acceptance Test Matrix before submitting the system/device/driver to test and document the test results in this matrix. Prior to submitting updated drivers, IHVs should review OCA and Watson crashes caused by their GPS drivers and fix all frequent crashes. Antenna Requirements from OEM 1. Managed systems with GPS support shall be tested according to the Cellular Telecommunications & Internet Association (CTIA) CTIA Test Plan for Mobile Station Over-the-Air Performance, Method of Measurement for Radiated RF Power and Receiver Performance v3.0+ for A-GPS and shall pass it. Total Isotropic Sensitivity (TIS), Upper Hemisphere Isotropic Sensitivity (UHIS) and Partial Isotropic GPS Sensitivity (PIGS) will be measured and OEMs shall post measurement results to Microsoft for review. 2. The system must have freespace TIS and UHIS of -140 dbm or better for GPS. The measurements must follow the test methodology and test parameters defined in CTIA 3.x test plan. 3. The average gain of the GPS antenna should be better than -6dBi. 4. Human hands holding the device in one of the common handgrips should not cause performance to drop below the minimal acceptable standard. Device should be able to maintain OTA acquisition sensitivity at -140 dbm and OTA tracking sensitivity of -145 dbm when the system is held in common positions. 5. Antenna performance and radiated sensitivity testing should be performed when the GPS antenna is in their expected positions within the device. 6. EV systems should have the intended GPS antenna for production added in its intended location for production and DV systems should have antenna location finalized. 7. OEMs should run antenna performance and radiated sensitivity tests and understand the failures prior to sending EV units. The tests must pass prior to DV units. Operator Certification Tests If operator certification tests for A-GPS are executed, pass/fail results and waiver information is to be shared with Microsoft. Dependency on Third-Party Services or Win32 applications There shall be no dependency on third-party services or Win32 applications accompanying the GPS solution. Dependency on third-party services has power implications, hence not allowed. Third-party Win32 applications are subject to signing requirements on SoC systems and not allowed.
GNSS (GPS) Test Guidance - 9 USB Selective Suspend (if applicable) USB connected GPS devices shall support selective suspend. Firmware Updates GPS on mobile broadband modules shall update via UEFI and standalone GPS shall update via driver. Test Descriptions Functionality WHCK tests that apply to GNSS devices are the first set of tests to verify the basic functionality of GPS devices. WHCK contains tests for GPS Sensors, Radio Manager, Device Fundamentals, System Fundamentals Power Management, Windows Driver Test Framework (WDTF), and USB Hardware Certification tests (for USB connected devices) that apply to GNSS devices. More information about WHCK tests can be found at http://connect.microsoft.com/. The functionality tests are intended to run under clear sky conditions. 1. Sensor category, type, properties and data fields Description: The device should report correct sensor category and type and support mandatory properties and data fields. In addition to the mandatory sensor properties documented in MSDN, speed data in knots (SENSOR_DATA_TYPE_SPEED_KNOTS) and heading data relative to true north (SENSOR_DATA_TYPE_TRUE_HEADING_DEGREES) are mandatory for managed program systems. Execution Steps: Query sensor category, type, properties and data fields the Device Under Test (DUT) reports. Sensor Diagnostics Tool (SDT) in Windows Driver Kit (WDK) can be used to test this. Expected Result: Mandatory fields must be supported and report correct data. 2. State Transitions Description: The device should report changes in sensor state, as documented in MSDN. Data reports should only be reported after device reaches to SENSOR_STATE_READY or SENSOR_STATE_INITIALIZING. Device should not report data if it does not have lat/long information, e.g. time only. GPS sensor should start in SENSOR_STATE_INITIALIZING. GPS sensor should continue to acquire fix and stay in initializing state until the request is cancelled by the OS.
GNSS (GPS) Test Guidance - 10 GPS Sensor should go into SENSOR_STATE_READY when it gets a fix and has data. GPS Sensor should go into SENSOR_STATE_INITIALIZING when it loses the signal and no longer has data. It should go back to SENSOR_STATE_READY when it reacquires fix. Execution Steps: Disable and re-enable the device. Monitor state transitions and data events. Move into an area with no GPS signal (e.g. faraday cage), wait for at least one minute, and move back into coverage area. Monitor state transitions and data events. Expected Result: Device should report sensor state transitions e.g. from initializing to ready. Data reports should only be reported after reaching to initializing and ready states. Data should not be reported if latitude and longitude information is not available. Device should start in initializing state and should not move into ready state before getting a fix and has a valid error radius. When moved out of GPS signal coverage area, the device should go into SENSOR_STATE_INITIALIZING and it should go into SENSOR_STATE_READY when moved back to coverage area. 3. Accuracy of Latitude and Longitude Description: The device should provide accurate latitude and longitude values within the specified error radius. Execution Steps: During static testing and in vehicle testing, the data from device under test (DUT) will be compared to latitude and longitude data reported by reference GPS, survey markers and simulator. Expected Result: The difference between the latitude and longitude values reported by DUT and the reference GPS should be within the error radius DUT reported. 4. Minimum Report Interval Description: GPS driver should not send data reports more frequent than current report interval. Execution Steps: Query the current report interval. Monitor the time of the previous and latest data reports. Compare the time between data reports to current report interval. Expected Result: Device should not send data reports more often than current report interval. 5. Report Interval Set Description: GPS driver shall support setting the report interval. Device should be able to report data at 1Hz frequency. It should calculate the minimum report interval in case of multiple clients setting the report interval to different values. It should reflect changes from clients going away and new clients coming in and setting report interval. Driver should follow the guidelines in MSDN managing multiple clients setting report interval.
GNSS (GPS) Test Guidance - 11 Execution Steps: Set report interval from multiple clients. Query the current report interval. Monitor the time of the previous and latest data reports. Compare the time between data reports to current report interval. Add new clients changing the report interval, remove clients who previously set the report interval. Report intervals to set: 1 seconds 15 seconds 1 minute Expected Result: Device should be able to report data at 1Hz frequency. Device should support setting report interval and should correctly reflect changes in the report interval from multiple clients. It should keep report interval up to date with new clients coming in and going away. It should not send data reports more often than current report interval. 6. Report Interval Delay Description: GPS driver should not delay data reports more than 100% of the currently set report interval when it is in ready state. Execution Steps: Query the current report interval. Monitor the time of the previous and latest data reports. Compare the time between data reports to current report interval. Expected Result: Device should not delay data reports more than 100% of the currently set report interval. 7. Speed Data Description: DUT should report speed data in knots when moving. Execution Steps: Perform simulated in vehicle tests, manual walking tests and drive tests and monitor the speed data reported by device. Expected Result: Device should report speed data, and accuracy of the speed information should be within ±10% of the speed data reported by reference GPS or simulator. 8. Heading Data Description: DUT should report heading in degrees relative to true north when moving. Execution Steps: Perform simulated in vehicle tests, manual walking tests and drive tests and monitor the heading data reported by device. Expected Result: Device should report heading data, and accuracy of the heading degrees should be within ±10% of the speed data reported by reference GPS or simulator.
GNSS (GPS) Test Guidance - 12 9. Other Sensor Properties Description: If other sensor properties are supported, they should report valid data and accurate values. Execution Steps: Monitor properties supported by DUT and ensure they provide valid data within acceptable accuracy. Expected Result: If a sensor property is supported, they should report accurate values within ±20% of the reference GPS or simulator reported values. 10. Radio State and Flight Mode Description: Turning flight mode on or turning GPS radio off from Windows Control Panel should turn off the GPS radio. Device should go into lowest possible power state when radio is off. Device should be able to acquire fix after turning the radio back on. Execution Steps: While a location client application is active (e.g. sensor diagnostic tool), turn flight mode on or GPS radio off. Monitor data and state change events from DUT. Turn radio back on or turn off flight mode. Try to acquire fix. Expected Result: Driver should stop reporting data events and change sensor state to SENSOR_STATE_NOT_AVAILABLE. Device should go into lowest possible power state. After turning the radio on or turning the flight mode off, it should be able to acquire fix. 11. Basic PnP Description: Uninstalling and re-installing the GPS driver, disabling and re-enabling the device should work as expected. Execution Steps: While a location client application is active (e.g. sensor diagnostic tool), disable and re-enable the device. Attempt to acquire fix. When device is in use again, uninstall the driver and then reinstall. Expected Result: Device should be able to get disabled and re-enabled. It should be able to acquire fix after enable. Driver should be able to get uninstalled gracefully. Assisted GPS When the GPS is first powered up, it should be able to use assisted GPS to return an approximated location in a few seconds. When using assisted GPS, the sensor should provide location data, which can be several hundred meters to six figure numbers. Later as the GPS radio is able to obtain multiple satellite locks, the error radius should decrease to a value between 3 30 meters. 1. A-GPS Description: A-GPS should help getting a faster TTF, with lower accuracy. Execution Steps: Cold start the GPS device. Monitor latitude, longitude and error radius data fields from Sensor Diagnostics Tool (SDT). Based on the following conditions:
GNSS (GPS) Test Guidance - 13 Clear-sky conditions (simulated or actual) Subscribed for data events Report interval of one second Wi-Fi or cellular baseband is present and enabled Expected Result: Device should return a position from A-GPS as soon as possible and associated error radius should be reported. The higher error radius (e.g. 300 meters if Wi-Fi is available) should decrease to 3-30 meters as device acquires multiple satellite locks. GPS should report a position based on assistance data within 15 seconds. 2. Position Injection GPS driver can use data from triangulation sensors present on the system to speed up Time To First Fix (TTFF) using Sensor API. Robustness a. Connection Time Description: GPS driver should close the connection to other sensors immediately after getting the position. It should timeout after 15 seconds and closes the connection to Sensor API if it does not get a position. Execution Steps: Start monitoring the traces from Sensor API for the active client counts for all sensors in the system. Cold start the GPS device. Monitor the changes on the active client counts for other sensors in the system. Expected Result: If active client counts for other sensors are incremented, they should return to their previously recorded values after the 15 seconds. b. Connection Type Description: GPS drivers should not instantiate ILocation to get data from other location sensors. They can use the Sensor API (ISensorManager) to open connection to for instance triangulation sensors (SENSOR_TYPE_LOCATION_TRIANGULATION). GPS driver should not get data from location sensors of the same type. For instance, a GPS sensor should not use data from other sensors with type GPS to get a faster fix. Execution Steps: Find out the sensor type the DUT is reporting e.g. SENSOR_TYPE_LOCATION_GPS. Disable all sensors except the sensors of the same type with the DUT. Start monitoring the traces from Sensor API for the active client counts for enabled sensors in the system. Cold start the GPS device. Monitor the changes on the active client counts for sensors in the system. Expected Result: Active client counts for the sensors of the same type should not be incremented. Driver Verifier, WDF Verifier and Application Verifier will be enabled for location platform and GPS device stack during all testing to test the reliability of the GPS support in the system.
GNSS (GPS) Test Guidance - 14 1. Driver Verifier, WDF Verifier and Application Verifier Description: Enable Application Verifier and Driver Verifier at the beginning of testing. Execution Steps: Enable Driver Verifier on all kernel mode drivers in the driver package (if any) and Application Verifier for %windir%\system32\wudfhost.exe for WDF drivers at the beginning of testing. Enable Driver Verifier on WDF drivers and any kernel mode drivers you have. Application Verifier (appverif.exe) is available in WHCK Client and SDK. A minimum of basic settings without Leak detection is required. Driver Verifier is part of the OS builds. It can be launched from admin command prompt with following settings for kernel mode drivers: Verifier /standard /driver wudfpf.sys Wdf01000.sys Wdfldr.sys wudfrd.sys <any kernel mode driver you have> WDFVerifier is enabled by default for all WDF drivers. WdfVerifier.exe tool in WDK can be utilized to control the verbosity of logging, debugger settings etc. Expected Result: No verifier crashes. 2. Telemetry Data Description: Monitor telemetry data from Microsoft device web for the GPS driver. Execution Steps: Monitor telemetry data from Microsoft device web for the GPS driver. Identify, investigate and fix the crashes in the driver. Expected Result: All telemetry reported crashed should be triaged and the top crashes should be investigated and fixed. 3. GPS State Transitions Description: System will be cycled through 5 transitions of entering and exiting environment where a GPS signal can and cannot be received. GPS sensor state transitions will be verified among the ready and initializing states as expected. Execution Steps: 5 transitions of entering and exiting environment where a GPS signal can and cannot be received. Expected Result: GPS state should change to/from ready and initializing as expected. Device should be able to get a fix after these iterations. 4. GPS Stress Tests A combination of the operations listed below will be performed simultaneously on the GPS device during simulator scenario execution, walk testing and drive testing: Enable Driver Verifier Enable Application Verifier Repeated WHCK test run (GPS Sensor, Radio Manager, System Power Management)
GNSS (GPS) Test Guidance - 15 Manual operations on Sensor Diagnostics Tool Manual Radio Management Operations Manual Connected Standby Manual GPS device disable/re-enable. Windows Location Provider disable/re-enable Mobile Broadband or Wi-Fi device disable/re-enable Large download over Mobile Broadband connection Large download over Wi-Fi connection Bluetooth activity A basic verification test will be performed before the stress testing. The expectation is that the same basic verification test to pass after the stress and no crashes are observed during testing. Performance GPS device performance will be tested for cold start TTFF, hot start TTFF, acquisition sensitivity, tracking sensitivity, re-acquisition time, static navigation accuracy, and dynamic navigation accuracy. GNNS Simulator will be utilized for performance testing. 1. Cold Start TTFF Description: Cold Start TTFF should be achieved in less than 45 seconds 90% of the time. Cold start is described as: Time unknown Current ephemeris unknown Position unknown Execution Steps: Clear the GPS assistance data before the test. Ensure the cold start conditions described above. Monitor the TTFF under clear sky conditions (actual or simulated). Expected Result: Device should acquire a fix using the GNSS device within 45 seconds 90% of the time. 2. Hot Start TTFF Description: Hot Start TTFF should be achieved within 2 seconds. Hot start is described as: Time is known Almanac is known Ephemeris is known
GNSS (GPS) Test Guidance - 16 Position within 100 km of last fix Execution Steps: Ensure the hot start conditions described above. Monitor the TTFF under simulated clear sky conditions. Expected Result: Device should acquire a fix using the GNSS device within 2 seconds 90% of the time. 3. Acquisition Sensitivity Description: Device should be able to obtain a fix at -150 dbm or lower power levels. Execution Steps: Under simulated lab conditions, with a direct RF connection when antenna connector is accessible, expose device to low power levels up to -150 dbm. Expected Result: Device should acquire a fix at -150 dbm. 4. Tracking Sensitivity Description: Device should be able to maintain a fix at -155 dbm or lower power levels. Execution Steps: Under simulated lab conditions, with a direct RF connection when antenna connector is accessible, after obtaining a fix, reduce the power level down to -155 dbm. Expected Result: Device should be able to maintain a fix -155 dbm. 5. Reacquisition Time Description: Device should be able to reacquire fix within 2 seconds. Clear sky conditions assumed when signal is available. Execution Steps: Under simulated lab conditions, after obtaining a fix, reduce the power level down enough to get the DUT lose the fix. Then increase the power levels back and monitor the reacquisition time. Alternatively, drive through a tunnel during drive testing. Expected Result: Device should be able to reacquire fix within 2 seconds. 6. Static Navigation Accuracy Description: Device should be able to report accurate latitude, longitude and if supported, altitude. Execution Steps: Under simulated ideal lab conditions, compare the accuracy of longitude, latitude and if available, altitude to the simulated location. Expected Result: Device should be able to report horizontal accuracy of 15 meters and vertical accuracy of 30 meters at 95% of the time. 7. Dynamic Navigation Accuracy Description: Device should be able to report accurate latitude, longitude and if supported, altitude when mobile.
GNSS (GPS) Test Guidance - 17 Execution Steps: Under simulated ideal lab conditions, simulate in vehicle motion and compare the accuracy of longitude, latitude and if available, altitude to the simulated values. Expected Result: Device should be able to report horizontal accuracy of 15 meters and vertical accuracy of 100 meters. Power Consumption Battery life run down testing should be performed by evaluating battery life while executing predefined workloads described in ADK. The following are the scenarios that will be used to evaluate system power performance for GPS. GPS in use with clients subscribed at default report interval, high accuracy of data requested. Hourly lookups Location platform enabled. All previously connected clients disconnected. Location platform enabled. Flight mode on. Location platform disabled. 1. No GPS Clients Description: When there are no GPS clients connected to the location platform or subscribed for data update events, GPS device should go into D3, and be in the lowest possible power state. When there is a connected client but it s not subscribed for data update events (single shot or polling), GPS should acquire a fix and then enter low power tracking mode until client asks for a position. Execution Steps: Close all client applications (e.g. SDT). Monitor device power state from device manager (Device Properties/Details/Power Data). Launch a client app that only performs single shot look ups and does not subscribe for data events. Perform single shot look ups between long time intervals while moving and monitor the time to fix and accuracy of the data. Expected Result: GPS device should go into D3 when there are no connected clients. Device should go into low power tracking mode after single shot look ups. 2. GPS Radio OFF When GPS radio is turned off, GPS device should go into the lowest power state possible but not remove itself from the bus. 3. Radio State and Flight Mode and Power Management Description: The GPS driver needs to incorporate the radio state in power management handling. Device should go into lowest possible power state when radio is off. When active client count goes to 0 when radio is off, device should go into D3. When a client connects when radio is on, Device should re-enter D0, however, should only fully power on when the radio is enabled.
GNSS (GPS) Test Guidance - 18 Execution Steps: While a location client application is active (e.g. SDT), turn flight mode on or GPS radio off. Monitor data and state change events from DUT. Close client application (e.g. SDT). Monitor device power state (e.g. from Device Manager: Device Properties/Details/Power Data). Launch the client application again. Monitor device power state. Turn radio back on or turn off flight mode. Try to acquire fix. Expected Result: When radio is turned off, device should stop reporting data events and change sensor state to SENSOR_STATE_NOT_AVAILABLE. Device should go into lowest possible power state. After closing the client application, device should go into D3. When client application is re-launched, device should go back to D0, however should not fully power on. After turning the radio on or turning the flight mode off, it should be able to acquire fix. 4. USB Selective Suspend This test applies to USB connected devices only. GPS device with no clients subscribed for a report interval of 8 seconds or less should participate on selective suspend when all devices on the bus is ready to go into suspend mode. Device Manager and ETW events will be used to monitor the USB bus state transitions. Power Management 1. Connected Standby New Driver Verifier tools for Runtime Power and Location Platform Plug in for Windows Driver Test Framework (WDTF) will be utilized for Connected Standby (CS) testing. Driver Verifier Run Time Power tests will be enabled throughout the certification testing with the exclusion of power consumption and performance tests. a. Connected Standby with Active Clients Description: Put system into Connected Standby state when there are active clients and device is in ready state. Device should go into Connected Standby. Resume in coverage area after 30 seconds. Device should be able to acquire fix and report data within 2 seconds after resuming. Execution Steps: When there are active clients connected put the machine into connected standby. Wake from connected standby. Expected Result: Device should go into Connected Standby mode. After resume, it should reacquire fix and start reporting data within 2 seconds. b. Resume in no Coverage Area Description: Put system into Connected Standby state when there are active clients. Resume in no coverage area. Device should try to acquire fix and enter initializing state. Execution Steps: When there are active clients connected put the machine into connected standby. Wake from connected standby in an area without GPS signal. Expected Result: Device should try to acquire fix and go into initializing state.
GNSS (GPS) Test Guidance - 19 c. Connected Standby WHCK Tests with Sensor Active Clients Description: Start a GPS client application which is subscribed for data events and run System Fundamentals Power Management WHCK Tests. Execution Steps: Start a sensor client application which is subscribed for data events and run System Fundamentals Power Management WHCK Tests: System.Fundamentals.PowerManagement.CS.* Expected Result: Tests should pass and GPS device should get a fix and continue to function as expected at the end of the test. Antenna Performance 1. Performance Tests with OTA Connection It is common to do performance testing of GNSS receivers in lab environment over a cabled RF connection, bypassing the GPS antenna and associated circuitry. This approach does not give a complete picture of real-world device performance and issues in GPS antenna and its circuitry can cause poor user experience in location based service applications. To catch these issues, managed system device should be tested for GPS performance with an over-the-air (OTA) test methodology. Managed systems with GPS support shall be tested according to the Cellular Telecommunications & Internet Association (CTIA) CTIA Test Plan for Mobile Station Over-the-Air Performance, Method of Measurement for Radiated RF Power and Receiver Performance v3.0+ for A-GPS and shall pass it. Total Isotropic Sensitivity (TIS), Upper Hemisphere Isotropic Sensitivity (UHIS) and Partial Isotropic GPS Sensitivity (PIGS) will be measured. OEMs shall share results with Microsoft and let Microsoft know of any waiver scenarios. The system must have freespace TIS and UHIS of -140 dbm or better for GPS. The measurements must follow the test methodology and test parameters defined in CTIA 3.x test plan. The average gain of the GPS antenna should be better than -6dBi. Human hands holding the device in one of the common handgrips should not cause performance to drop below the minimal acceptable standard. Device should be able to maintain OTA acquisition sensitivity at -140 dbm and OTA tracking sensitivity of - 145 dbm when the system is held in common positions. In addition to antenna design and human interference from holding the device, device form factor and location of the GPS antenna also impact the antenna pattern. Therefore, antenna performance and radiated sensitivity testing should be performed when the GPS antenna is in their expected positions within the device. 2. Human Interference Tests Antenna positioning should take human interference into account. When the system is held in common states, GPS should not lose fix, should not increase the error radius
GNSS (GPS) Test Guidance - 20 more than 30%, and should be able to maintain OTA acquisition sensitivity at -140 dbm and OTA tracking sensitivity of -145 dbm. Commonly used states for slates: Hands on the sides. Landscape orientation Hand on bottom, Landscape Orientation Hands on the sides. Portrait orientation (Start on Left) Hands on the bottom. Portrait orientation (Start on Left) Hands on the sides. Portrait orientation (Start on Right) Hands on the bottom. Portrait orientation (Start on Right) Interoperability a. Human Interference Impact on Sensor State and Error Radius Description: The sensor state and error radius should not cause performance to drop below the minimal acceptable standard when it s hold at specified handgrips. Execution Steps: Under clear sky conditions, when device is in ready state, hold the device at commonly used states. Check sensor state and error radius. Expected Result: Device state and error should not cause performance to drop below the minimal acceptable standard. Device should not lose fix and increase the error radius more than 30%. b. Human Interference Impact on Acquisition and Tracking Sensitivity Description: The device acquisition and tracking sensitivity should not cause performance to drop below the minimal acceptable standard when it s hold at specified handgrips. Execution Steps: Hold the device at commonly used states. Check acquisition sensitivity and tracking sensitivity. Expected Result: Tracking and acquisition sensitivity should not be impacted when the device is held on specific locations. Device should be able to maintain OTA acquisition sensitivity at -140 dbm and OTA tracking sensitivity of -145 dbm. 1. Mobile Broadband, Wi-Fi, Bluetooth, Camera Interoperability Other radios and devices such as camera in the system can cause interference with GPS. It is also common that GPS device share the same module with mobile broadband, Wi-Fi and Bluetooth. GPS functionality should not be impacted by usage of these devices.
GNSS (GPS) Test Guidance - 21 Description: Simultaneous usage of Mobile broadband, Wi-Fi, Bluetooth and Camera should not degrade the performance and functionality of the GPS device. Active usage of GPS should not impact the functionality of these devices. Execution Steps: Run basic functionality tests with MB, Wi-Fi, Bluetooth and Camera on and actively in use. Large download over Mobile Broadband connection when using GPS. Monitor SDT and the event logs from SDT. Large download over Wi-Fi connection when using GPS. Monitor SDT and the event logs from SDT. Bluetooth file transfer while using GPS. Monitor SDT and the event logs from SDT. Wi-Fi scan while using GPS. Monitor SDT and the event logs from SDT. MB scan while using GPS. Monitor SDT and the event logs from SDT. BT scan while using GPS. Monitor SDT and the event logs from SDT. Video recording while using GPS. Monitor SDT and the event logs from SDT. Watch movie over internet (e.g. Netflix) while using GPS. Monitor SDT and the event logs from SDT. Expected Result: Simultaneous usage of these devices should not impact GPS and these devices functionality. GPS should continue to function when these devices in use. If these devices are functioning well without GPS being in use, they should continue to function well when GPS is in use, too. Drive Tests Manual drive testing where the system is taken on a drive on a route defined earlier which includes tunnels and areas with different multipath impact. During the drive, the system GPS data will be captured by a test application and will be compared to a reference GPS. At no time should the location reported by the system +/- the error radius be outside the location reported by the reference GPS. In addition, the average error radius should be <= 30 meters. Drive testing will exercise real life conditions i.e. dynamic navigation accuracy, reacquisition after going through a tunnel, impact of multi path and atmospheric conditions. Following functionality tests will be run during drive testing: State transitions will be monitored and compared to reference GPS. Latitude, Longitude and if available altitude will be monitored and compared to reference GPS. Speed and heading data will be monitored and compared to reference GPS.
GNSS (GPS) Test Guidance - 22 Simulator Tests Reacquisition time after going through the tunnels will be measured and compared to reference GPS. Tracking sensitivity will be monitored in multipath impact areas with regards to reference GPS. Dynamic navigation accuracy will be monitored and compared to reference GPS. Device PnP and radio manager state transitions will be performed during dynamic navigation. A GNSS simulator (Spirent GSS6700) will be utilized to achieve controlled lab conditions, replaying same test scenarios for repeatability, simulating satellite states, various locations and time e.g. south of equator and 2 years later, simulate in vehicle navigation, atmospheric conditions, multi-path and error conditions. Over The Air RF connection will be used to be able to test the system with original antenna and shielding. Direct connection may also be used when antenna connector is accessible for receiver testing. Simulator testing will focus on the common simulator scenarios including GNSS receiver performance characteristics and special scenarios: COLD Start TTFF HOT Start TTFF Acquisition Sensitivity Re-acquisition Sensitivity Tracking Sensitivity Static Position Accuracy Dynamic Position Accuracy Dynamic Position Accuracy Multipath GPS and GLONASS The standard scenarios supplied with the Spirent GSS6700 simulator will be run. Reporting and Results Communication All issues found will be communicated to partners via bugs. The feedback on the test categories listed above and their sub-categories will be provided in the form of a color coded score card. Each test area will be represented with a scorecard and the goal is having all scenarios green.
GNSS (GPS) Test Guidance - 23 We will also provide the test logs e.g. WHCK logs, traces, logs from partner driver and any relevant performance results and baseline performance comparison data. Test Equipment Toolset Resources Following test equipment will be used: Spirent GSS6700 GNSS Simulator Faraday Cage RF shielding box MB SIM Reference Devices External antennas WHCK Driver Verifier WDF Verifier Application Verifier Sensor Diagnostics Tool: WDKToolRoot\<architecture>\sensordiagnostictool.exe Mandatory sensor properties documented in MSDN: http://msdn.microsoft.com/enus/library/windows/hardware/ff545919(v=vs.85).aspx Guidelines in MSDN managing multiple clients setting report interval: http://msdn.microsoft.com/enus/library/windows/hardware/hh706203(v=vs.85).aspx http://msdn.microsoft.com/enus/library/windows/hardware/hh706201(v=vs.85).aspx. How to turn WDF Verifier on: http://msdn.microsoft.com/enus/library/windows/hardware/ff556127(v=vs.85).aspx other useful info on WDF Verifier: http://msdn.microsoft.com/enus/library/windows/hardware/ff556129(v=vs.85).aspx Application Verifier: http://msdn.microsoft.com/enus/library/windows/hardware/ff538115(v=vs.85).aspx Driver Verifier: http://msdn.microsoft.com/en-us/windows/hardware/gg487310