Analyzing rideshare data

The process of building the quantitative backbone for an app

Original app description

Over the past decade, ride-hailing services such as Uber, Lyft, Gett, Juno, and Via have revolutionized the transportation space. However, profitability for drivers varies across all these platforms in complicated ways. Drivers are forced to make mental calculations based on the time of day, surge rate, wait time to pick up a rider, etc. for each driving service, and choose to ride for a given service at any point based on the optimization of such factors. As a result of this difficult optimization problem — given the number of potential factors as well as the many unknowns — the driver has no certainty that they have chosen the most profitable service with which to ride. Ride King eliminates such uncertainty by pooling historical and to-date data for each driving service based on the driver’s location as well as the time at which the driver is looking to drive. The ultimate goal with Ride King is to eventually extend this optimization service to riders, such that they may be able to select the cheapest rideshare company with which to ride at any given point. The convergence of the respective optimizations for riders and drivers will hopefully detract enough from each rideshare service to engender a better sense of transparency and/or greater payouts for drivers.

Ride King: Ride Intelligently

A look at the raw data

The data collection for this app idea was by no means straightforward. To avoid having their information used for competitive purposes, most rideshare companies will do everything in their power to protect their data. However, with some deliberation, we can actually discover a couple backdoor methods to access these data. All rideshare apps in NYC are under the Taxi and Limousine Commission (TLC) and are required by law to submit data on every ride that occurs. With some investigation, the keen observer will find that TLC is part of the NYC Open Data initiative, which was established to increase transparency in state and city affairs. While the data these rideshare companies do provide are incredibly shallow (minimal information per ride), we can still pull quite a bit of information from them. To start us off, however, I‘m using Uber data from April 2014 (provided generously by the company, fivethirtyeight ) that have a slightly different format to the TLC dataset.

From a quick glance, we can see that each row in this set provides the date, time, location of pickup (in geographic coordinates), and ‘base’. We will get to the last column a bit later—for now, let’s just look at the raw data. Please note that, while we could just plot the raw geographic coordinates, I’ve actually overlaid NYC neighborhood boundary data that was generously made public by Zillow. This now has added another dimension to our data!

Okay, cool. So we can now see all the Uber pickups from April 2014 in Manhattan (zoomed, for readability) with neighborhood boundaries and all! Since the goal of this project is to try and predict the profitability of a rideshare service based on the time of day and location, we’ve only scratched the surface so far, but we can already see that rides are definitely not distributed evenly!