Yikes, a month and 10 days gone...

jasonrohrer · 2015-03-30 19:56:27

Looking back on my original post about a launch contest... wow, that was on February 20.

We've gone round and round, and it seems like there's no structures that's not either vulnerable to collusion or burdened with too many nasty anti-collusion side-effects.

In my original post, I was kinda speculating that it probably wasn't a solvable problem, but willing to punt for a solution one more time.

Well, that seemingly harmless punt turned into quite a long excursion down the rabbit hole.... and yeah, it doesn't seem like there's a bottom down here.

If the threat model is one person with 25 visa gift cards... it would be really hard to stop them from racking 25 easy wins while holding the amulet (5000 points) without an opponent-matching system so onerous that it would ruin the play experience for non-colluding amulet holders.

Even with "lots of active players," the reality is that active games take a while to finish, so the colluder has way more control over timing than I thought they did. Random delays, etc., only help a little bit when the colluder can play many games in rapid succession, always putting them in the right place at the right time. Meanwhile, non-colluders are still in the middle of their real games, unavailable as opponents for the amulet-holder.

In order to make it very likely that the amulet-holder plays a non-colluder, we must have the amulet holder wait a long time for one of these "real" players to become available (assuming a threat model where the colluder can't completely swamp the game with 50%+ fake accounts, picking a random active player and waiting for their availability means we pick a non-colluder most of the time).

So, it seems best to kill this idea and move on.

I've got all my wax carving supplies here, and I still like the idea of unique amulets as prizes.

As we discussed long ago, almost all of the "most X during the week" metrics are easily exploitable by multiple accounts or bots. Less exploitable metrics, like most unique opponents, are still exploitable by bots (not to mention frustrating for human players who can't control who the uniqueness of their opponents anyway).

A very boring option is just a straight-up random drawing for everyone who plays at least X games during the week. No purchase necessary, of course (would have to have a mail-in option for people to enter for free).

Maybe there are less boring options here...

This is a back-to-the-drawing-board moment for me.

On the bright side, it's good to realize this before the launch contest and not during or after. Which was the point of these test contests. There's no point in a test if the answer is known.

THANK YOU CARAVAN DISTURBER.

(You are aptly named.)

claspa · 2015-03-30 21:20:08

Wow, it really shows courage to be able to kill an idea and move on. You have my respect.

---

I came up with a new idea, it's not the best but maybe it will help you, smart people, to spark some great ideas out of it.

What about using a bit of the Swiss Tournament Style?

You can decide to play an Amulet Game. Within a timer of 5 minutes a player pool is formed.

Players will get matched who have the similar ranking and have not if possible played before.

If you win a game your Amulet ranking gets a +1, if not you stay at your ranking level.

---
That is the basic idea. Not spectacular, but we can work on that.

CaravanDisturber · 2015-03-31 01:06:18

Bummer...

What about a contest that takes into account fresh ELO ratings just for the contest? Horrible playing accounts would be worth less with each game they lose. Then a Gold Silver Bronze prize from the initial buy-in. I guess that's basically the tournament.

joshwithguitar · 2015-03-31 01:07:40

I can understand the desire to abandon it all at this point but I'm not sure it would be that hard to fix with a few comprises and a different approach.

I think you ran the second tournament much too fast. After the first we outlined that there was lots of potential collusion exploits. You added some hasty patches and quickly ran another test. If instead you'd posted the rules and the server code and asked "how do you exploit this" you'd have had a large list of problems from us within 24 hours. Many of the problems had already been noted before the comp. The only reasons exploits didn't come up faster in the test is because we were being fair.

If we ran through a number of fixes and threw them to the players to figure out an exploit I don't think it would take long to get to something workable - partly because I think we are almost there.

So, currently there seems to be two lines for exploitation:
maximising amulet pickups and therefore points
maximising amulet games - hopefully against your colluders

The first is very easy to deal with. Get rid of the 200 points given when you pick up an amulet. It shouldn't happen too often anyway. This completely removes any chance of this exploit being a thing.

The second is more tricky. The only way I can think of doing the exploit is this:
play lots of games against colluders where one player leaves/loses quickly
lose to any colluding amulet holder

The only way I can see for doing this is:
- chose an unpopular stake
- have two accounts join the stake at the same time
- if both join each other one leaves
- if one is left waiting treat the other game as an amulet game.
- if brought into a game with a colluder, lose

Now, if this is the only exploit available then it seems there are lots of ways of dealing with it, many of them simple:
- have it so that facing the same opponent within X minutes disqualifies that matchup from drawing into an amulet game.

- have it so that after the start of any game a player will not be considered for amulets games for 5 mins (new games during this period will not reset the timer though).

- put a cap on the number of games per day that once passed makes players no longer eligible for further amulet games that day. (this would also restrict the power of bots and people with far too much spare time).

These are just a few but they all seem to do the job on their own and I'm sure there are plenty more.

The other thing I'd recommend is this to basically make it so the contest isn't running - no amulet games occur - if the player base gets too low at any point in the day. Hopefully this won't happen very often. But it is a simple fact: the idea really only works if there are enough active players that any player who wants to can start a game within a couple of minutes.

Anyway, I hope I have managed to make you rethink things, but if not good luck with thinking up something new!

CaravanDisturber · 2015-03-31 02:53:16

joshwithguitar wrote:

if brought into a game with a colluder, lose

I don't understand how it would know, that it's two buddies playing?

If Jason knew that i'm pretty sure you could just block them from playing.

I think the fix for amulets, it's complicated and might not be worth it.

1. Use maxmind IP filtering and block IP addresses from the same region of the world. Easier option would be just to graph a users playing times, to find their "Sleep Period" or Time. You might also be able to base it on the Zip code used when registering, purchasing coins. (Region could be based on a miles cap.) (But you should always be allowed to play NORMAL GAMES with buddies on the same network. It's actually pretty thrilling to play against buddies in person.)

2. This wouldn't work for buddies playing across regions. In order to solve that, you could use a few options to filter. Email Domains, IP address connections, etc. (Google Analytics for Apps has some cool data too.)

3. Drastically increase the user pool for the amulet to drop. Seems like the math: 2*N + 3 is solid for bare min. That's not a normal metric. I would adjust to (2*n)+(x/8)+3 or some variation. Where x = 5 day rolling average users per day. Hopefully the math makes colluding next to impossible. (the 8 comes from .333 of 24 hours)

4. Drastically increase the number of people playing the same, which would make #3 work better.

Also would be nice to have referral codes for inviting friends where users get rewarded $.25 or something per referral. You could then assume they know who referred them or who they referred.

Last edited by CaravanDisturber (2015-03-31 02:55:18)

jasonrohrer · 2015-03-31 03:59:18

Well, there's no way to stop two buddies from playing each other completely. The problem is that if they exclusively play each other repeatedly to gain loads of false points. Playing each other once is okay.

But multiply "two buddies" into 25 false accounts, which is easy to achieve with a few trips to the gas station for visa gift cards.

With that in mind, Josh, I though about it for many hours this morning before I finally made that post. I worry that all of your fixes can be bypassed by using more than just two accounts. Yes, fixing the "drop amulet and repeat" stuff is easy enough. But fixing the threat of someone with the amulet playing against dummy accounts is really hard.

Remember, you only fire up your dummy accounts when you get the amulet that you want. That's a rare event. Maybe a once-in-the-contest event. So, those dummy accounts would have no strikes against them. They wouldn't have been playing too many games already, or playing X games per minute.

The other problem is that, as the amulet-holder, you control the timing. You decide to play an amulet game and when to play it. That means your colluding accounts have an advantage over any other account in getting paired with you, no matter what we do, for the simple fact that they know WHEN to try the most and no one else does. They're not going to be in the middle of another game. They're not going to be on lunch break.

So... you create an amulet game... maybe you wait for a moment when there are very few other games waiting. Then you start your pairs of dummy accounts joining each other. To bypass any of your limits, just use each pair of dummy accounts once. If you had 24 accounts, that'd be 12 chances in a row. Or if Josh's first suggested limit is in place, tracking only pairs, have one dummy account cycle through playing all 23 others in a row.

Then you play a 5 minute game against the dummy (to bypass any short-game blocking methods), and also to reset all of your timers, Josh, so that the 12 dummy pairs could be ready to go again.

Unless your timers are extremely long (like, longer than the 2-hour force-drop window), the amulet holder could play a waiting game with a stable of dummy accounts and rack up unlimited points. And putting really long limits in place for pairs of players would create a frustrating situation for other players (playing the same person repeatedly at some stake is pretty normal---why am I never put into an amulet game??)

And if you did the obvious thing of blocking an amulet-holder from winning too many points from a given non-amulet account... well, having multiple dummy accounts obviously bypasses that.

The one fix that would work is to pick the opponent for the amulet-holder ourselves, based on no timing that they or their colluders could control (storeroom's suggestion). But that means the amulet holder would have to wait for the chosen opponent to be ready to play, which would be a pretty terrible experience for the amulet holder. Imagine the case where the chosen opponent is done playing for the day, so they're never ready to play the amulet-holder. There'd be some timeout (15 minutes was suggested) and then we'd have to pick someone else and wait AGAIN, making the wait even worse for the amulet-holder.

My spouse had a really funny, perverse suggestion today: A contest with the tagline "May the best cheater win."

joshwithguitar · 2015-03-31 06:38:52

So the only big worry now is one person creating lots of alt accounts or a large group of people helping to prop up each other.

The second seems unlikely and the first seems more likely and tricky to do without gift cards. I think I mentioned this before but couldn't you ban gift cards from creating accounts? Use something like address verification to check if it is one?

In both cases we are talking about very overt and very noticeable behaviour. You won't need much detective skills to figure out what is happening. Why not now just say, ok, I am going to make a rule against collaboration that is totally at my discretion. Anyone who I deem to have used extensive collaboration to win an amulet will be forfeit the prize. I don't see any problems with this and think in the end it will stop anyone from doing anything with 25 accounts. And if they do, people will notice, there will be records and they won't get anything. All without much effort needed from you at all, though I think it will act as enough of a deterrent that it will require basically no effort.

Cobblestone · 2015-03-31 08:28:46

Unfortunately, there's two problems with banning Visa gift cards: one is that they allow you to create and register an account with your address on file for when you have to use it for a site that requires address verification... and the other is that, given the nature of the game (an anonymous, occult themed betting game), you'll probably be turning away real players who want to use a gift card for legitimate reasons (since I almost did that until I read up on the CC encryption etc for the game).

I going to toss out a really dumb idea I had, because why not. If the issue is collusion between the amulet holder and non-amulet holders, then maybe the solution is to get rid of any interaction between them. I know you liked the idea of passing the football around, but what if amulets were only given to a player out of the pool to the winner of a random non-amulet game for 0 points. After that you only pair two amulet games together. The winner keeps and gets points for their amulet, and the loser drops theirs back into the pool. This is probably exploitable too, but maybe less so? I just thought I'd throw it out there.

Whatever you decide for the contest, I think the idea of the amulets for prizes is awesome, and I'm glad you're sticking with that.

Dan_Dan84 · 2015-03-31 14:24:52

I'm sad to hear that The amulet passing idea might be gone, but glad we still might have a chance at winning a cool amulet.

Since we're back in "random ideas" teritory again...

Perhaps you could divide the amulets in half: copper ones can be won the "boring way," but gold and silver ones can only be won through a more "tried and tested" tournament structure. I know you wanted to avoid anything resembling a tournament, but this structure would give everyone a chance at winning something, but only more skilled players could win a more valuable amulet.

Anyway, just a thought.

jere · 2015-03-31 16:14:52

The one fix that would work is to pick the opponent for the amulet-holder ourselves, based on no timing that they or their colluders could control (storeroom's suggestion). But that means the amulet holder would have to wait for the chosen opponent to be ready to play, which would be a pretty terrible experience for the amulet holder. Imagine the case where the chosen opponent is done playing for the day, so they're never ready to play the amulet-holder. There'd be some timeout (15 minutes was suggested) and then we'd have to pick someone else and wait AGAIN, making the wait even worse for the amulet-holder.

The random delay/timing stuff has always struck me as very exploitable. And you're right that picking a specific opponent and having the amulet holder wait for them is pretty bad. Here's an idea that I think is better: queue players up and randomly assign the queued players every, say, 5 minutes. The timer is shown on the UI (e.g. next assignment in 1:20).

-You start/join a stake.
-You know exactly how long you'll have to wait for a game. This'll let you know if you have time to take a quick bathroom break or whatever.
-If 20 people join (and there are <= 10 amulet holders), they all get games.
-Sometimes you will be the odd person, but then you'll be guaranteed a game on the next assignment (assuming the game isn't completely dead).

This has some reasonable worst case bounds on waiting and it hurts colluders pretty hard. Their chance of getting a colluding game is proportional to how many accounts they control over how many total non-amulet holders there are (not just at a given instant that the colluder controls but over a FIVE MINUTE span). It'd hurt even more if amulet holders were allowed to be assigned to each other. If you want, you could require a minimum number of players in the queue before pairing them.

Another idea that may have been thrown out there that I will agree on: make points proportional to how long a game lasts based on time or rounds (with a cap around 10-20 minutes). What Caravan says they were doing (going all in immediately) isn't how most games play out realistically and this diminishes the effect of fast colluder games.

Both of these ideas prevent colluders racking up points very fast, but shouldn't bother normal players too much. One other thing about the points being proportional to time: it somewhat punishes this atypical, reckless play that you see quite often during the amulet contest. You still can get points for doing it but they'll be worth less.

Last edited by jere (2015-03-31 17:17:06)

claspa · 2015-03-31 17:19:24

jere wrote:

The random delay/timing stuff has always struck me as very exploitable. And you're right that picking a specific opponent and having the amulet holder wait for them is pretty bad. Here's an idea that I think is better: queue players up and randomly assign the queued players every, say, 5 minutes. The timer is shown on the UI (e.g. next assignment in 1:20).
-You start/join a stake.
-You know exactly how long you'll have to wait for a game. This'll let you know if you have time to take a quick bathroom break or whatever.
-If 20 people join (and there are <= 10 amulet holders), they all get games.
-Sometimes you will be the odd person, but then you'll be guaranteed a game on the next assignment (assuming the game isn't completely dead).

Thank you for explaining what I tried to do.

claspa wrote:

You can decide to play an Amulet Game. Within a timer of 5 minutes a player pool is formed.
Players will get matched who have the similar ranking and have not if possible played before.
If you win a game your Amulet ranking gets a +1, if not you stay at your ranking level.

I think it has some potential.

jasonrohrer · 2015-03-31 17:31:05

Enforcing slower games is interesting, but also weird. Some games are legitimately faster. We wouldn't want to encourage slow playing (people would be motivated to do that to get more amulet points). Finally, though it would slow down colluders, it still wouldn't deal with the fundamental issue (unlimited number of points to be had with no skill).

I've also thought about queuing.... but does that mean the non-amulet people would be waiting around too? Like, they would know they were in the queue and be watching a timer too?

Waiting for certain people to be ready to play seems fragile and messy.

I end up thinking in terms of a kind of false conception of what's going on. Thinking of all these amulet-holders waiting for games, and all these other people joining games, and if we mark certain people as queued, they'll just pair up, and there will be nice bounds on wait times. BUT, the reality is that most of the time, people are in-game, with no bound on how long that game will last. In fact, a closer picture to reality is one where there's never more than one person waiting for a game (the odd person), and there aren't really loads of new games starting up all the time.

The false conception is "chaos, with a flurry of activity." But even with quite a few more players, that chaos isn't usually realized.

jere · 2015-03-31 18:18:19

BUT, the reality is that most of the time, people are in-game, with no bound on how long that game will last. In fact, a closer picture to reality is one where there's never more than one person waiting for a game (the odd person), and there aren't really loads of new games starting up all the time.

I see what you're saying, but that's only because games are joined on demand (the small exception being the random delay stuff, which was easily bypassed).

If you instead cast a net and scoop up all the players that show up in a 5 minute period, then that will be a flurry of activity... at least it really should be during the contest. I'd like to see stats on how long your average game lasts, but if it were around 20 minutes, this queue should average around 20% of the player base. If it's a dead time and you don't get more than, let's say, 4 players queued, just extend the timer another 5 minutes. It's not like people never have to wait for games currently....

Basically, what we currently have is a network of taxis/cars that shuttle you off as soon you want to go. What Claspa and I are suggesting (and apologies for missing his post) is analogous to a subway that arrives on set times. In the current situation, the bank robbers keep escaping in their getaway cars and we keep puzzling over how to change the speed limit.

does that mean the non-amulet people would be waiting around too? Like, they would know they were in the queue and be watching a timer too?
Waiting for certain people to be ready to play seems fragile and messy.

Everyone within the appropriate stakes are queued. There's no need to select specific people. In fact, the benefit here is that you pick from the most people possible. Everyone sees a timer. I can't comment on how messy that is, but this seems like one of the best collusion-busters outside of a straight up tournament.

Last edited by jere (2015-03-31 18:32:04)

jasonrohrer · 2015-03-31 19:30:37

Yeah... but it also seems to change the game quite a bit for the non-amulet holders.

CaravanDisturber · 2015-03-31 21:06:22

I still think your ELO ratings solve the collusion. Just need to figure out how to work that in. If accounts constantly get Losses added to them, they are less valuable when you beat them. However, If you beat a quality opponent you should a better score.

Cobblestone · 2015-03-31 21:28:43

CaravanDisturber wrote:

I still think your ELO ratings solve the collusion. Just need to figure out how to work that in. If accounts constantly get Losses added to them, they are less valuable when you beat them. However, If you beat a quality opponent you should a better score.

I was suggesting this yesterday in the chat and a few points were brought up to counter this, unfortunately. Colluding accounts could just pass the amulet around between them, alternating wins and losses and ranking up points simultaneously.

Ghost Amount · 2015-03-31 21:44:57

I am leaning much more towards storming Caravan's home with pitchforks and torches.

Maybe I am just angry because he beat me twice yesterday.

CaravanDisturber · 2015-03-31 22:00:13

Cobblestone wrote:

CaravanDisturber wrote:
I still think your ELO ratings solve the collusion. Just need to figure out how to work that in. If accounts constantly get Losses added to them, they are less valuable when you beat them. However, If you beat a quality opponent you should a better score.
I was suggesting this yesterday in the chat and a few points were brought up to counter this, unfortunately. Colluding accounts could just pass the amulet around between them, alternating wins and losses and ranking up points simultaneously.

Hmmm..

Solid Points, However ELO removes points for a loss. It's weighted based on account skill. The biggest issue with this would be the Top ELO scorers. They have a higher win expectancy, Thus earn less points with a win over a horrible account. However they are much more likely to win more than one match.

What if the above worked with along with No drop button. And when you lost an amulet match, your account was frozen from joining an amulet game again for a set period of time? There has to be something there...

CaravanDisturber · 2015-03-31 22:01:13

Ghost Amount wrote:

Maybe I am just angry because he beat me twice yesterday.

Who didn't beat you twice yesterday?

CaravanDisturber · 2015-03-31 22:04:00

1. ELO Ratings
2. Random Amulet that is Dropped.
3. Adjusted Amulet drop rate.

I think would solve most collusion.

Adjusted Amulet Drop Rate:

CaravanDisturber wrote:

3. Drastically increase the user pool for the amulet to drop. Seems like the math: 2*N + 3 is solid for bare min. That's not a normal metric. I would adjust to (2*n)+(x/8)+3 or some variation. Where x = 5 day rolling average users per day. Hopefully the math makes colluding next to impossible. (the 8 comes from .333 of 24 hours)

computermouth · 2015-04-01 00:28:11

CaravanDisturber wrote:

1. ELO Ratings

I would probably just never get a match. A loss leading player who works nights and weekends, leaving me morning and superlate nights. During amulet tourneys, I spent at least 80% of my time in queue. Add any filtering to it and it's practically hopeless for me without a significantly larger player pool.

jasonrohrer · 2015-04-01 01:33:41

It turns out that Elo is pretty exploitable with multiple accounts. Yes, the amount you win by beating the same opponent over and over goes down over time (as your Elo rises and theirs falls), BUT it never goes to zero. So accomplices and bots can achieve whatever Elo they want by playing hundreds of games and getting a little Elo boost each time.

Of course, there's little motivation to "cheat" the Elo leaderboard as it stands. But when prizes are involved, there's plenty of motivation.

CaravanDisturber · 2015-04-01 01:59:19

jasonrohrer wrote:

It turns out that Elo is pretty exploitable with multiple accounts. Yes, the amount you win by beating the same opponent over and over goes down over time (as your Elo rises and theirs falls), BUT it never goes to zero. So accomplices and bots can achieve whatever Elo they want by playing hundreds of games and getting a little Elo boost each time.
Of course, there's little motivation to "cheat" the Elo leaderboard as it stands. But when prizes are involved, there's plenty of motivation.

The Math might need to be different, IE: No basement and most importantly, it needs to not be published. The number ratings that is. Only the standings should be published.

Also, I've noticed with some standings the where numbers are the same their isn't a same place. IE: ELO.

3. Fake Name 272
4. Caravan Distuber 272

May it be unpublished decimals?

joshwithguitar · 2015-04-01 13:41:43

I'm going to state this one more time as I really want to hear your response Jason and it was kind of buried before:

I think there is a very simple solution which along with some minor tweaking will make the contest ready to launch. Just write a rule against colluding/using alts to gain easy amulet points or help pick up amulets. Have this rule entirely at the discretion of Jason and have it so that any player deemed to have broken the rule forfeits any prizes.

Now all we need to do is prevent small scale collusion being a problem (refer back to my earlier post).
And you should be set as any useful collusion should be obvious, and as such will be avoided.

Jason, I know you have been very resistant to this idea but it has been over a month and you are on the brink of giving up. I really think this could get things working very soon and am concerned that nothing else will work. Perhaps it's time to reconsider these things?

LiteS · 2015-04-01 15:35:54

joshwithguitar wrote:

I think there is a very simple solution which along with some minor tweaking will make the contest ready to launch. Just write a rule against colluding/using alts to gain easy amulet points or help pick up amulets. Have this rule entirely at the discretion of Jason and have it so that any player deemed to have broken the rule forfeits any prizes.

I think there are laws against contests setup like this, specifically to avoid cases where someone in Jason's position colludes with his friends. At this point the contest might as well have the rules of every other sweepstakes. "Deposit money into CM, one entry per person, randomly chosen winners. Multiple entries are disqualified, no purchase necessary, mail here for alternate entry."

CORDIAL MINUET ENSEMBLE

#1 2015-03-30 19:56:27

Yikes, a month and 10 days gone...

#2 2015-03-30 21:20:08

Re: Yikes, a month and 10 days gone...

#3 2015-03-31 01:06:18

Re: Yikes, a month and 10 days gone...

#4 2015-03-31 01:07:40

Re: Yikes, a month and 10 days gone...

#5 2015-03-31 02:53:16

Re: Yikes, a month and 10 days gone...

#6 2015-03-31 03:59:18

Re: Yikes, a month and 10 days gone...

#7 2015-03-31 06:38:52

Re: Yikes, a month and 10 days gone...

#8 2015-03-31 08:28:46

Re: Yikes, a month and 10 days gone...

#9 2015-03-31 14:24:52

Re: Yikes, a month and 10 days gone...

#10 2015-03-31 16:14:52

Re: Yikes, a month and 10 days gone...

#11 2015-03-31 17:19:24

Re: Yikes, a month and 10 days gone...

#12 2015-03-31 17:31:05

Re: Yikes, a month and 10 days gone...

#13 2015-03-31 18:18:19

Re: Yikes, a month and 10 days gone...

#14 2015-03-31 19:30:37

Re: Yikes, a month and 10 days gone...

#15 2015-03-31 21:06:22

Re: Yikes, a month and 10 days gone...

#16 2015-03-31 21:28:43

Re: Yikes, a month and 10 days gone...

#17 2015-03-31 21:44:57

Re: Yikes, a month and 10 days gone...

#18 2015-03-31 22:00:13

Re: Yikes, a month and 10 days gone...

#19 2015-03-31 22:01:13

Re: Yikes, a month and 10 days gone...

#20 2015-03-31 22:04:00

Re: Yikes, a month and 10 days gone...

#21 2015-04-01 00:28:11

Re: Yikes, a month and 10 days gone...

#22 2015-04-01 01:33:41

Re: Yikes, a month and 10 days gone...

#23 2015-04-01 01:59:19

Re: Yikes, a month and 10 days gone...

#24 2015-04-01 13:41:43

Re: Yikes, a month and 10 days gone...

#25 2015-04-01 15:35:54

Re: Yikes, a month and 10 days gone...

Board footer