So a story in Businessweek caught my eye the other day. It discussed NYC taxi rider tipping habits and concluded that riders usually tip between 20% and 25% using the histogram below:
Their plot showed bumps at 20%, 25% and 30%, the default tip options on the credit card readers in cabs, which made sense. But something else did not feel right to me about the plot. For the people not using the default buttons, why did so many more end up tipping 21% than 19%? Is every rider rounding up? Curious.
As I looked more at the article, things kept unraveling a bit. Another chart in the article showed that at 4PM, people suddenly started tipping 1% more.
For an explanation of the evening tip uptick, the reporters reached out to Bhairavi Desai, executive director of the New York Taxi Workers Alliance. Her explanation of this increased tip in the article said:
“During rush hour, people appreciate the privacy and convenience of a taxi more than being packed in a subway,” Desai says. “Most of the riders who take taxis in those hours are regulars, who tend to talk more to drivers and have a more sensitive understanding of driver economics.“
Once again, this explanation felt pretty strange to me. A solid one percent gain felt too large for this behavioral response. So I decided to dig in a bit.
The first step was to figure out how to calculate a tip percentage. The TLC data has many cost fields:
- Fare Amount
- Surcharge – used during peak times
- MTA Tax – fifty cent tax on most in-city trips
To calculate a tip percentage, the numerator was obviously going to be the tip amount. But what about the denominator? What are we tipping 20% on top of? Should it be just the Fare Amount? Fare Amount plus Surcharge? Or should we be tipping on Taxes and even Tolls?
To find out how the in-cab payment systems calculate tip, I took a sample of taxi rides (with toll payments) and tried two different options:
- Option 1: Default Tip = (Fare + Surcharge) * 20%
- Option 2: Default Tip = (Fare + Surcharge + Taxes + Toll) * 20%
I made a table with these two different ways of calculating tip, and I also colored round numbers in green:
So for this table, each row has one column with a green box. There is always either a round number in the tip column (most likely punched in by a rider) or a round value in the option 2 column (indicating choosing a percent, as opposed to a dollar amount). This means that the in-taxi payment system is calculating tips including taxes and tolls!
I was pretty surprised by this, but then it got stranger.
Here is another sample I looked at:
Notice in this sample, Option 1 is much more prevalent in Green than Option 2. Meaning these riders (with the exception of two) seemed to be given defaults not calculated using tax and toll.
So what gives? Well, notice the first field in each table. It turns out that when it comes to calculating tips, payment systems provided by Creative Mobile Technologies (CMT) are programmed differently than those provided by Verifone (VTS). Since this data was from 2013, I looked at a few of my own recent cab ride receipts to verify that the issue is still present.
The receipt on the left from RidLinQ (the payment app for CMT) shows the same 20% tip amount of $3.20 as the receipt on the right from Way2Ride (the payment app for Verifone). But the base fares in green are different. Here, CMT is calculating tips including tax, giving the driver the same default tip for less mileage as the Verifone driver.
So what does this mean? Well in short cab drivers who are driving CMT programmed cars are making more money in tips from New Yorkers than those driving Verifone programmed cabs. (About half of all NYC cabs have CMT payment systems.) To estimate the total size of excess tips, I looked at the subset of CMT processed credit card rides where users used default tips. The excess tips in 2013 amounted to about $2.0 million in tips from tolls and $3.2 million in tips from the $0.50 MTA tax on most standard rides, for an annual total of about $5.2 million in extra tips. That’s about $200 in extra tips on average for each unique driver of CMT enabled cabs per year!
Now back to the Businessweek article. If we redo their histogram, but use two different “tip” calculations depending on the vendor, we get a very different picture:
That feels a lot better and more in line with intuition. It turns out that 46% of users pick the default 20% option, 11% choose the 25% option and about 4% choose the 30% option. So about 60% of rides are affected by this programming difference.
And how about the exploration and quotes in the Businessweek article pertaining to increased tipping during rush hour? Well, there is a surcharge from 4PM to 8PM, exactly in line with the increase observed by the Businessweek article. That surcharge would cause the default tips as a percentage of base fare to go up, and thus cause the boost in tips discussed in the article. Redoing the analysis with corrected data yields a different result:
So how was Bhairavi Desai able to explain the increase in evening rush hour tips that did not exist? Well, Desai’s explanation and the story’s analysis are an example of what I like to call “post rationalization”. Most of the time, given data, an intelligent person can come up with a reasonable explanation for that data. But this is exactly what the scientific method is supposed to prevent. Best practices would dictate that you first come up with a hypothesis, and then look at the data to back it up.
Unfortunately, a lot of data science can’t follow that method because data scientists (or data journalists) don’t always have a hypothesis at the ready. They first look at the data, like the Businessweek article did. Sure enough, most of my work on this blog does not start with a hypothesis, though some does (like my hypothesis that combined fast food restaurants are dirtier than non-combined ones.) So I am often left post rationalizing my own findings. As we see in this example, post rationalizing needs to be done with extra care and plenty of caveats.
The TLC should fix the in-cab payment systems to make them consistent with one another. Rider’s should not have to know which payment system they are using when deciding whether to use a pre-calculated tip or not. While I am not weighing in on whether to include tax and tolls in the calculation of tip, consistency and a clear explanation is what is warranted and fair, both for riders and for drivers whose income should not be dependent on the company processing payments in the back of their cab. I truly hope that they fix this rapidly. The DOT set a great example fixing the misleading fire hydrant I exposed quite quickly but the MTA has refused to apply a quick fix for the vending machine issues I wrote about in a prior post. I’m hoping the TLC follows the DOT approach.
The TLC should release taxi data directly to the public, not through FOIL. Releasing data allows the TLC to learn more about its operations from the very citizens it serves. Unfortunately, the agency continues to take the FOIL approach instead of the Open Data approach. In this particular case, it was Chris Whong who FOILed the data and posted it online, not the TLC itself. If the TLC released data to the public via Open Data, this might have been found and corrected sooner.
When Doing Data Science, Look at Your Raw Data. If there is one thing I have learned doing data science, it is to always look closely at your raw data in addition to your aggregate statistics. It would not have been possible to figure this out without looking at a subset of individual rides.
Link To or Publish Your Data Sources. If data science is in fact a “science”, it should be reproducible. The ability to check someone else’s work is EXACTLY why using public data is important. I’ve had readers of this blog note some of my own mistakes along the way. That’s precisely the point. So big kudos to the authors of this Businessweek article for linking to the public data underneath their work. It was their article that inspired this analysis and helped uncover the peculiarities of the in-cab taxi payment systems. Along those lines, news agencies should be publishing their underlying FOILed data along with there analysis for this very reason. Some, like the New York Times Upshot, have started to do so.
Update: Eagle eyed reader @dtimesd pointed out that this has been the case since at least 2009. He points to Page 4 in a recent academic paper called “Default Tips” by Haggag & Paci where two mysterious NYC taxi payment vendors called “Vendor” and “Competitor” engage in this practice. I guess we now know who Vendor is and who Competitor is thanks to public data.
- justdirtandash likes this
- davepander likes this