Making Sense of Urban Data

Transcription

1 Making Sense of Urban Data Anil Yazici, PhD Camille Kamga, PhD Data Hasselt, Belgium July 213

2 NEW YORK CITY Population, 211 estimate: 8,244,91 Land area in square miles, 21: Persons per square mile, 21: 27,12.5

3 TRANSPORTATION MODE IN NEW YORK CITY Transit Systems: Rail Commuter Subway Systems (NYCT, PATH) Bus Systems Ferries Private Automobile Taxi Dollar Vans Bicycle Pedicabs Walking

4 GPS-TAXI IN NEW YORK CITY More than 13,2 Taxis 27 Credit Card Payment and GPS system

5 What is urban data like?

6 Data flow while you are walking to the subway Corner of St. Nicholas Av. & W145th St. February 2 nd, ~5:2 PM

7 Smart Cities, Urban Science Popular topics both in industry and academia IBM Cisco Accenture CUSP (Center for Urban Science and Progress) at NYU-Poly Media Lab at MIT Not confined to transportation Energy consumption, carbon footprint, waste management, emergency management Basically, tackling all types of urban challenge with help of data sources

8 TLC Yellow Cab Data GPS dataset with more than 37 million taxi trips covering the period from January 1, 29 to November 28, 21. Each trip record includes: trip origin and destination (Lat-Long) the time of pick-up and drop-off the number of passengers trip fare (base fare + surcharge + tolls ) trip distance

9 How to make sense of taxi data? Probe vehicle travel time congestion patterns Travel time variability/reliability Passenger urban travel patterns Home to work, or business to another business? Morning rush hour vs. Evening rush hour Taxi as a taxi Taxi industry, a multi billion $ business Taxis role in public transportation system Taxi demand and supply Medallion prices Fare hikes New technology: E-hail?

10 Any Drivers? Sunday Saturday Friday Thursday Wednesday Tuesday Monday Sunday Saturday Friday Thursday Wednesday Tuesday Monday Sunday Saturday Friday Thursday Wednesday Tuesday Monday 1AM 2AM 3AM 4AM 1AM 2AM 3AM 4AM 1AM 2AM 3AM 4AM 5AM 6AM 7AM 5AM 6AM 7AM 5AM 6AM 7AM 8AM 9AM 1AM 8AM 9AM 1AM 8AM 9AM 1AM AVERAGE TRAVEL TIMES ACTUAL AVERAGE TRAVEL TIMES 11AM 1PM 2PM 3PM VARIANCE OF TRAVEL TIMES 11AM 1PM 2PM 3PM 4PM 5PM 6PM 4PM 5PM 6PM COV OF AVERAGE TRAVEL TIMES 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM STANDARD DEVIATION OF TRAVEL TIMES 7PM 8PM 9PM COEFFICIENT OF VARIATION OF TRAVEL TIME DISTRIBUTION 7PM 8PM 9PM 1PM 11PM 1PM 11PM 1PM 11PM

11 Weather impacts? What happens to your travel time when it is raining? What about travel time variability? WEATHER DATA Data gathered from Weather conditions are updated every hour, unless there is a change in the existing conditions. For each trip, the weather at the time of pick-up is assumed as the weather condition for the entirety of that particular trip.

12 Assumptions

13 Percentage Change in Travel Time - Light Rain PERCENTAGE CHANGE IN AVERAGE TRAVEL TIMES - LIGHT RAIN Sunday Saturday Friday Thursday Wednesday Tuesday Monday 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM 2% %

14 Percentage Change in Standard Deviation - Light Rain Sunday Saturday PERCENTAGE CHANGE IN STANDARD DEVIATION - LIGHT RAIN 5% 4 Friday 3 Thursday 2 Wednesday 1 Tuesday Monday 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM -1 %

15 Percentage Change in Coefficient of Variation - Light Rain PERCENTAGE CHANGE IN COEFFICIENT OF VARIATION - LIGHT RAIN Sunday Saturday Friday Thursday 25% Wednesday Tuesday Monday 1AM2AM 3AM 4AM5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM6PM 7PM 8PM 9PM 1PM 11PM -5-1 %

16 Travel Time, Congestion Why do we (or should we) care? Actual vs. Perceived travel time Objective vs. Subjective travel time Travel decisions; time, mode Value of travel time (VOT) Toll pricing Economic evaluation of transport projects Value of travel time reliability (VOR) Mode choice, route choice and more

17 How is the Value of Travel Time/Reliability Calculated? Stated Preference Surveys Revealed Preference Surveys

18 Subjective vs. Objective Travel Times Subjective.. based on what? Individual Time One, first, needs to know the objective travel times, to investigate the subjective travel times A guideline for objective travel time patterns are needed Taxi GPS data with almost 4 million records provides good basis for objective travel times

19 Classification and Regression Trees (CART)

20 Homogeneous Time Periods Based on CART Monday Tuesday Wednesday Thursday Friday Saturday Sunday ESTIMATED AVERAGE TRAVEL TIMES 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM 4 6 Monday Tuesday Wednesday Thursday Friday Saturday Sunday ESTIMATED STANDARD DEVIATION 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM Monday Tuesday Wednesday Thursday Friday Saturday Sunday ESTIMATED COV 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM Monday Tuesday Wednesday Thursday Friday Saturday Sunday ESTIMATED AVERAGE TRAVEL TIMES 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM Monday Tuesday Wednesday Thursday Friday Saturday Sunday ESTIMATED STANDARD DEVIATION 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM Monday Tuesday Wednesday Thursday Friday Saturday Sunday ESTIMATED COV 1AM 2AM 3AM 4AM 5AM 6AM 7AM 8AM 9AM 1AM 11AM 1PM 2PM 3PM 4PM 5PM 6PM 7PM 8PM 9PM 1PM 11PM

21 Making Theoretical Sense : Travel Time Reliability Measures The concept of reliability in general is based on component failures relatively well defined for mechanical and production systems. For transit systems with scheduled trips, failure can be defined based on on-time performance for road networks, there is no such concept. Many measures are suggested for road networks Variance, coefficient of variaation, measures based on distribution quantiles, etc Let s see whether these measures (supposedly measure the same phenomenon) are consistent

22 - 1AM 1AM - 2AM 2AM - 3AM 3AM - 4AM 4AM - 5AM 5AM - 6AM 6AM - 7AM 7AM - 8AM 8AM - 9AM 9AM - 1AM 1AM - 11AM 11AM 12PM 12PM - 1PM 1PM - 2PM 2PM - 3PM 3PM - 4PM 4PM - 5PM 5PM - 6PM 6PM - 7PM 7PM - 8PM 8PM - 9PM 9PM - 1PM 1PM - 11PM 11PM Consistency of Suggested Measures 1: Unreliable to Reliable CoV Wed Fri Sun Skew Var CoV Skew Var CoV Skew Var Something is wrong? There is a need for more robust reliability measures

23 Making More Sense: Taxi Industry in New York City More than 13, licensed taxicabs Average of 66, passengers every day (Schaller Consulting, 24) There are 68, to 895, passengers based our findings. No obligation for taxi drivers to serve at any given time Lease prices vary based on the day of a week and day/night shifts.

24 Taxi Industry in General Taxi industry is heavily regulated (e.g., taxi operating permits and licenses). Once the taxi supply is regulated and a market is formed through taxi licenses, a taxi fleet becomes an economic entity. The decision to issue new licenses or to permit fare changes requires research.

25 What to look for? How do ridership and taxi-supply levels change for different time periods and under various weather conditions? Does weather have an impact on the passenger pick-up rate and on hourly revenues for taxi drivers? What are the possible impacts of the recent New York City taxi-fare increase on taxi demand and supply equilibrium?

26 WHY? UTRC s JFK Taxi Study Drivers don t come to JFK for pick up during rain. They make more money in Manhattan when it rains All cabs are full!! A Manhattan resident s cry during rain Urban legend? Individual taxi operators are income-target oriented and often reduce the amount of service as the demand increases (Schaller, 1999; Thaler, 1997) What if not 17 percent? Lease prices, medallion market

27 Taxi Ridership and Trip Statistics in NYC Mon Tue Wed Thu Fri Sat Sun Average Ridership 679,29 762,55 787, , , , ,566 Average Number of Trips Average Occupancy (passengers/taxi) 42, , ,5 516, ,846 55,79 419, NYC subway and bus weekday average ridership are about 5.2 million and 2.5 million respectively.

28 Average Ridership Average Number of Drivers Average Ridership and Average Number of Drivers on Duty Per Hour 5 x 1 4 Average Ridership per Day Per Hour -1AM 1AM-2AM 2AM-3AM 3AM-4AM 4AM-5AM 5AM-6AM Mon Tue Wed Thu Fri Sat Sun Average Number of Drivers per Day Per Hour -1AM 1AM-2AM 2AM-3AM 3AM-4AM 4AM-5AM 5AM-6AM x AM-7AM 7AM-8AM 8AM-9AM 9AM-1AM 1AM-11AM 11AM-12PM AM-7AM 7AM-8AM 8AM-9AM 9AM-1AM 1AM-11AM 11AM-12PM x PM-1PM 1PM-2PM 2PM-3PM 3PM-4PM 4PM-5PM 5PM-6PM PM-1PM 1PM-2PM 2PM-3PM 3PM-4PM 4PM-5PM 5PM-6PM 5 x 1 4 6PM-7PM 7PM-8PM 8PM-9PM 9PM-1PM 1PM-11PM 11PM- Time PM-7PM 7PM-8PM 8PM-9PM 9PM-1PM 1PM-11PM 11PM- Time

29 Average Number of Pick-ups Average Trip Distance [miles] 4 2 Average Distance per Trip and Number of Trips Per Hour per Driver Average Number of Pick-ups per Hour per Driver What about the revenue? 5 Average Trip Distance -1AM 1AM-2AM 2AM-3AM 3AM-4AM 4AM-5AM 5AM-6AM -1AM 1AM-2AM 2AM-3AM 3AM-4AM 4AM-5AM 5AM-6AM AM-7AM 7AM-8AM 8AM-9AM 9AM-1AM 1AM-11AM 11AM-12PM 6AM-7AM 7AM-8AM 8AM-9AM 9AM-1AM 1AM-11AM 11AM-12PM PM-1PM 1PM-2PM 2PM-3PM 3PM-4PM 4PM-5PM 5PM-6PM 12PM-1PM 1PM-2PM 2PM-3PM 3PM-4PM 4PM-5PM 5PM-6PM PM-7PM 7PM-8PM 8PM-9PM 9PM-1PM 1PM-11PM 11PM- Time Mon Tue Wed Thu Fri Sat Sun 6PM-7PM 7PM-8PM 8PM-9PM 9PM-1PM 1PM-11PM 11PM- Time

30 # of PICK-UPS DISTANCE [MILES] Average Trip Distance and Pick-up Rate per Driver per Hour under Different Weather Conditions Clear Light Rain Moderate Rain Heavy Rain Light Snow Moderate Snow Heavy Snow AVERAGE PICK-UPS PER HOUR PER DRIVER AVERAGE DISTANCE PER TRIP -1AM 1AM-2AM 2AM-3AM 3AM-4AM 4AM-5AM 5AM-6AM -1AM 1AM-2AM 2AM-3AM 3AM-4AM 4AM-5AM 5AM-6AM AM-7AM 7AM-8AM 8AM-9AM 9AM-1AM 1AM-11AM 11AM-12PM 6AM-7AM 7AM-8AM 8AM-9AM 9AM-1AM 1AM-11AM 11AM-12PM PM-1PM 1PM-2PM 2PM-3PM 3PM-4PM 4PM-5PM 5PM-6PM 12PM-1PM 1PM-2PM 2PM-3PM 3PM-4PM 4PM-5PM 5PM-6PM PM-7PM 7PM-8PM 8PM-9PM 9PM-1PM 1PM-11PM 11PM- TIME 6PM-7PM 7PM-8PM 8PM-9PM 9PM-1PM 1PM-11PM 11PM- TIME

31 DOLLARS [$] Revenue per Hour per Driver for Different Weather Conditions AVERAGE REVENUE PER HOUR PER DRIVER AM 1AM-2AM 2AM-3AM 3AM-4AM 4AM-5AM 5AM-6AM AM-7AM 7AM-8AM 8AM-9AM 9AM-1AM 1AM-11AM 11AM-12PM PM-1PM 1PM-2PM 2PM-3PM 3PM-4PM 4PM-5PM 5PM-6PM PM-7PM 7PM-8PM 8PM-9PM 9PM-1PM 1PM-11PM 11PM- TIME Clear Light Rain Moderate Rain Heavy Rain Light Snow Moderate Snow Heavy Snow

32 DOLLARS [$] DOLLARS [$] Estimated Impacts of Taxicab Fare Increase REVENUE PER HOUR PER DRIVER (5AM-6AM) % 17.4% 17.6% 16.7% % 17.% 16.8% 2 Clear Base Fare + Surcharges Light Rain Moderate Rain Heavy Rain Light Snow Moderate Snow Heavy Snow Metered Fare REVENUE PER HOUR PER DRIVER (2PM-3PM) % 15.4% 15.1% 15.1% 15.6% 14.6% 13.6% 2 Clear Light Rain Moderate Rain Heavy Rain Light Snow Moderate Snow Heavy Snow

33 What sense did we make? Substantial variations in ridership, trip distances and taxi supply for different DOW-TOD and weather-conditions. Minimum taxi supply is maintained at the level at which drivers receive approximately $2 per hour (excluding tips). Under rain conditions Higher pick-up rates and slightly shorter-distance trips Higher hourly revenues Snow conditions do not necessarily increase the hourly revenue per driver.

34 On Taxi Demand-Supply Balance Taxi activity mainly confined to Manhattan and the whole available taxi inventory is not on the streets even at peak taxi demand periods. Increasing the available number of taxis do not guarantee an increase the level of service, especially if there is no guarantee for drivers of certain minimum revenue (e.g. $2 per hour) It is unlikely that weather conditions change the taxi supply at the beginning of the day or shift. However, when there is prolonged inclement weather conditions, drivers who reach their income targets may end their shifts early, leading to temporary taxi shortages.

35 Possible Implications of the Latest Fare Increase After the latest the fare adjustments, revenue increases are not experienced equally for all DOW-TOD periods. the uneven revenue increases among different shift periods may eventually alter the corresponding lease prices. the increase in revenues may affect taxi drivers shift choice to be on duty.

36 More on Taxis In Progress Taxi driver decision making process for airport pick-ups After each dop-off, driver can either go to airport (decision=1), or keep on cruising for new passengers on the street (decision=) Binary decision Binary logistic regression Factors that affect drivers decision to go to JFK: Factors Time of Day Seriously? Day of Week Borough Weather Condition First Trip? Short Return Ticket

37 Quantifying the Impacts Factors Time of Day Day of Week Borough Weather Condition First Trip? Short Return Ticket Impacts ~1.5 times more likely during PM peak compared to AM peak ~¼ times less likely on Saturdays compared to Sundays ~5 times less likely at Manhattan compared to Brooklyn ~1/5 times less likely during rain ~ 4 times more likely if it is the first trip of the shift ~3 times more likely while at Queens and Brooklyn

38 OK, Quantified Then What? Policies?

39 More Ongoing Research No! Why do people take taxi? Why pay more when you have a dense subway system? Is the subway system really dense? Ask for someone living in Queens, or Red Hook How does the land use affect the taxi ridership?

40 The end Die Ende Fin Son Thank you! Q&A