Interview: Xbox Live's Stephen ToulouseBy Stephen Johnson - Posted Mar 04, 2010
A YouTube video that hit the web a week or so ago essays a conflict between a user of Xbox Live and a moderator. It kicked off a minor internet controversy, because in it, moderator The Pro seems to ban a player's console for no reason.
Soon,an unedited verrsion of the video hit the web, it tells a totally different story. It seems the banning went down after profuse and colorful profanity is hurled, and modding activity is clearly evident. Here's the totally not safe for for work, unedited video.
I spoke to Stephen Toulouse, the Director of Xbox Live Policy and Enforcement at Microsoft to get the inside word about moderation from the guy at the top of the mod heap.
G4: How does one become a moderator for Xbox Live?
Stephen Toulouse: To give you an overview of my role, it’s my job to run the team that handles policy enforcement for the entire Xbox Live service. And there’s three key pieces to that. There’s a core team that reports directly to me. And we’re responsible for writing policy, reviewing complaints and doing moderation. So for the core team, the Microsoft interview process is what determines that level of moderator.
The second piece is really a virtual team. These are full-time Microsoft Xbox employees who also help us in policing and moderating the system. They can handle specific issues like if there’s a gamertag or a profile that’s bad, they can escalate cheating issues up to my core team. And they’re all over the world. They must be full time Microsoft Xbox employees, and they must go through about an hour and half of training from my team directly in order to help guide them on how to help us moderate the system, to try and provide safe and enjoyable experience to the customers.
Their scope of ability is much narrower than my core employees. Hence they don’t go through a formal, Microsoft employee interview process. They don’t report directly to me, but their capabilities can have an impact. For instance, if one of them was abusing that power, that’s something we would handle with their management and remove them from the moderation.
And the largest, most important capability we have for moderation in the system is the customers, because they have the complaint ability within the Xbox to let us know when someone is behaving badly, and we can take whatever action we need to take to help correct that situation. So it’s not as simple as "how you become a moderator," as there are many many levels of moderator.
G4: How does the complaint system work?
G4: I’ve noticed that!
It requires a lot of training and continual discussion. We don’t want to take actions that tip the other way and make the service too authoritarian. The whole goal of Xbox Live is to be fun. And we need to step in when individuals are ruining that safe and enjoyable experience.
G4: How many mods are actually out there playing games?
ST: There’s quite a bit of being out in the system and actively policing. So we have a number of different gamertags that are communal, meaning a number of people could be using it, not at the same time. They rotate in and out. So we go out there and work to be the enforcement there in the game, but you don’t really know it’s an enforcement member. You could be gaming with these individuals and not know it.
We don’t seek to incite behavior or anything like that. If you watch the unedited video [of The Pro and Itzlupo] you can see quite a bit of what was cut out. So we don’t seek to incite, but we do have quite a few people out there gaming every day across the world, looking for that kind of behavior.
G4: Did the moderator The Pro do anything wrong?
ST: I spoke to him about his tone. I think we have to be dispassionate about issuing whatever corrective action we have to take. The individual who was using that gamertag was certainly regretful about the situation and his tone. But I want to stress: He was doing the right thing.
The punishment for the things these individuals were doing, which were modifying their consoles, opening up cheating lobbies for Prestige in Modern Warfare 2, and the really egregious vocal comments, yeah that results in permanent suspensions for their gamertags and consoles. So the actions were correct.
From a leadership standpoint, for my team, I want us to be looked at as the fair and objective moderators who are working to protect the service.
G4: Do you look for people who keep a cool head?
ST: We’re all human. While that’s a skill we look for, I think if you look at the unedited video, that’s probably going to be the behavior that’s going to provoke it.
We’re not interested in grudges or vendettas. That’s why we issue temporary suspensions far, far more than we issue permanent suspensions. We want people to realize there are rules out there, and learn from the experience and be better players. But when we have to take action, we will take action.
G4: Is there an appeals process?
ST: There is in a sense. We have a suspensions forum on Xbox.com where people can go to inquire about actions we take. We don’t see a lot of appeals because people look at that content and say, “Yeah, I shouldn't have put that in there.”
G4: How about when it’s something said or done in-game; Is that more of a gray area?
ST: Again, it’s pretty clear from our side. It’s only gray if on the customer side. Let me give you an example: If it’s a week after the launch of a game, and you suddenly pop into a lobby with your avatar having a completely different skin color than is allowed because you hacked your profile, and your gamerscore is 600,000, which is impossible, and you have a rank in that game that’s impossible to attain without 50,000 gamer hours, and then we hit you for cheating and system tampering, you probably know what you did.
The gray areas are where we try to have a light touch. If I’m in a room and I’m gaming, and someone gets shot from across the room and he blurts out a bad word, that’s not something I’m going to jump all over. I’m going to watch the tone of the room, see how people are behaving – are people getting upset, is this person repeatedly using that word, is it racist or homophobic speech? If none of those things are true then we’re going to continue on, and I’ll say, “Hey, I can appreciate that was frustrating, but maybe there’s kids in the room…” It would depend on the situation, and but we try to analyze the situation. We don’t say, “Oops, you just blurted a bad word. That’s it for you, Bye Bye.” We do try to keep it fun.
G4: But it still seems like a lot of Live players are pretty homophobic and racist.
ST: Xbox Live is subset of the internet. One of the great things about the internet is its ability to link people together in a common forum from all over, no matter what they think or where they are. The downside of that is, due to the law of large numbers, given the number of people on Live (over 23 million), that small subset of people who are racist or homophobic, it looks like it’s a larger number because even though it’s a tiny fraction of 23 million, it’s still a lot of people. It’s not that we’re telling them not to be racist. That’s not really our role. Our role is to protect the Terms of Service, and the Terms of Service are designed to protect the user experience,
So if someone comes in with that type of speech, we’re going to take action against them. I don’t expect from that action that they’re going to go off and suddenly become not homophobic – I would hope that they would – but in that case my job there is that they not use speech that is specifically prohibited, and that most people find very offensive.
G4: What's the ratio of mods to players?
ST: I don’t’ have exact numbers. At any given time there are dozens and dozens and dozens of people playing who are my moderators. And in addition we’re working seven days a week, 365 days a year going through the complaints from our most important moderators, the customers.
G4: How many complaints do you receive in a day?
ST: That’s tough to say. Right now, we’re at our lowest point, really, As I’m sure you could imagine, it peaks and ebbs when there’s a holiday or a big title comes out. Here’s something that has a really compelling affect on our complaint volume: When a really compelling single-player title comes out. Like, let’s say BioShock 2… even though it has a multiplayer component, most people are buying that for the single-player experience. So you’ll actually see this interesting dip because people are now going to play that single-player for a while, then they’re going to go back into the multiplayer world of Xbox Live.
At any give time it’s going to be different. It’s never, at any given time, gone over a fraction of one percent. Even on the worst days, it’s not over a fraction of one percent.
G4: Is there an inherent conflict of interest because banning a player essentially costs Microsoft money?
ST: No. The volumes just don’t work out to anything appreciable. As I’ve said, the vast majority of actions we take are temporary. Therefore, the user does come back, because it’s not a permanent suspension. if you’re asking: Is the bar set too high in the interest of revenue, the answer is no, because we don’t refund you for the account if you are suspended.
G4: But if you're perma-banned, monthy fees won’t come back.
ST: You’re correct, once they are permanently closed, but it depends on whether they want to open up another account and come back. But my team doesn’t really look at that as any deciding factor whatsoever. As far as we use the bar for permanent suspension, it’s things like fraud, it’s things like nudity on camera, those are instant permanent suspensions. Do not pass go.
And then there’s the repeat people. If we hit someone with temporary suspensions, over and over again, they do cross the line, where we have to make that a permanent suspension, but it’s not based on revenue. We don’t even see the revenue numbers from that perspective.
In the end, when I have to sit down and make the decision to do a permanent suspension, I’m making that decision for all the customers who are behaving more than the one who isn’t.
G4: What are the biggest misconceptions gamers have about Live moderation?
ST: I think hands down, that it’s automated. It’s absolutely not automated. There are a lot of people out there who believe that if you file enough complaints against someone, the system will automatically take action. We’ll see people sit there and file 300 complaints in an hour. What our system does is take those 300 complaints and makes it one complaint. Your gamertag vs. their gamertag for one infraction is one complaint. So my team never sees the 300. They see the one it’s collapsed to.
There’s no automated system where repeatedly complaining against one person is going to get them kicked off the system. And having your friends gang up on a person doesn’t work either, because we review the complaints for accuracy.