Spring 2010 MATLAB Contest, April 28 - May 5
in [Matlab]
|
Prev: Single number into vector elements
Next: neural network
From: Nathan on 6 May 2010 16:05 Hello all Congratulations to the prize-winners and thank you again, Mathworkers. I missed most of this contest but I had fun in the last few days, spectating and thinking (unsuccessfully) about the problem. This problem has the nice feature that it doesn't provide feedback to the solver (like Ants) and is probably the closest we've had yet to a real-world application (?). Alan: "But I do always abide by the explicit rules the contest team puts in place." The rules: "we ask that you not overwhelm the queue" Alan: "One of the tactics I tried yesterday morning was to ... fill the queue with lots of entries that take up almost the full 180 second limit ... those queue delay entries would significantly increase the time it took before other entries could be scored, artificially inflating my longevity time. " Was that not a deliberate effort (though misguided, on this occasion) to "overwhelm the queue" for advantage in a prize race? I don't know how hard it is to write code that talks to a web server, but I don't think it's a skill that this contest should reward. The contest has always been about raw abstract problem-solving ability, even more than it is about matlab. Problem-setters have always been skilfully avoided rewarding specialist knowledge (not even specialist knowledge of compressive sensing seems to have helped in this case). Specialist knowledge of web programming should not be a factor. Daylight, contrary to some comments in this thread, is not all about tuning of numerical parameters. Many contests have featured a sprinkling of algorithmic advances in daylight, even in the closing minutes. I doubt if there is any practicable way the organisers can forcibly eliminate the various methods that many competitors dislike, even if they were willing to. However there is a community ethos here that's stronger than our individual competitive streaks. There's lots of evidence for this, notably the respect for the ban on wholesale probing, and the fact that code scrambling has not has much impact. Could we embody this spirit in a kind of players' charter that states what the purpose of the contest is, and lays out (somewhat vague) principles of good conduct? Parts of the rules are already formulated like this - calls for fair play, rather than enforcable prescriptive rules. I've been thinking for a few contests about the nature of the Grand Prize, which currently is based on a snapshot of the scoreboard at a particular instant. This has motivated some great competition in the last minutes, and to win one of these is a big achievement. But the guy who wins the last stage of the Tour De France doesn't win the race. I think the most prestigious prize should recognise success at all stages of the contest - maybe by tallying up mid-contest prizes (with weightings and/or tie-break criteria) or by a multi-day "Push". The details would take a bit of working out - it's much easier to push on day 4 than day 7, and so on... Finally I'd repeat the request for a CAPTCHA, for the usual reasons, and echo Jan's request for a longer twilight. It seems a lot of people really enjoy that phase. See you all next time Nathan
From: Sergey Y. on 6 May 2010 19:44 "Alan Chalker" <alancNOSPAM(a)osc.edu> wrote in message <hrvd59$68n$1(a)fred.mathworks.com>... > Why do more than 2/3rds of people just submit a couple times instead of being involved more? I propose it's because they quickly find they are overwhelmed or unable to come even close to the expert players. How many of those even come back to subsequent contests? > > I think a good question to ask is why do we always see the same general set of names showing up as winners and what can be done to level the playing field for novices and experts alike so that the contest is more inclusive? > I believe it is possible to run some small challenges only for novice. For example 1000 node challenge is very suitable for this. It can be run on separate machine to avoid interference with main stream.
From: Darren Rowland on 6 May 2010 22:42 > > Why do more than 2/3rds of people just submit a couple times instead of being involved more? I propose it's because they quickly find they are overwhelmed or unable to come even close to the expert players. How many of those even come back to subsequent contests? > > > > I think a good question to ask is why do we always see the same general set of names showing up as winners and what can be done to level the playing field for novices and experts alike so that the contest is more inclusive? > > > @Alan. I am one of those 2/3rds of people who submit only a few entries early on. I then watch the results of the contest and see how the scores develop. This is partly due to time constraints and partly because the leading submissions by e.g. Jan and Oliver quickly progress beyond my understanding. This is why I have appreciated your "commented leading entry" during the past few contests, and also the mid-contest analysis. I had hoped that Steve Eddins would write one up for this contest, as he seems ideally qualified. > I believe it is possible to run some small challenges only for novice. For example 1000 node challenge is very suitable for this. It can be run on separate machine to avoid interference with main stream. @ Sergey. This is a good idea. A small contest for novice players (perhaps on the weekend) where a novice can be defined as a player who has never won a prize or has only submitted a handful of entries for the contest so far. I can imagine this would generate some interest for players like myself. Darren
From: Alan Chalker on 7 May 2010 00:24 "Darren Rowland" <darrenjremovethisrowland(a)hotmail.com> wrote in message > @Alan. I am one of those 2/3rds of people who submit only a few entries early on. I then watch the results of the contest and see how the scores develop. This is partly due to time constraints and partly because the leading submissions by e.g. Jan > and Oliver quickly progress beyond my understanding. > This is why I have appreciated your "commented leading entry" during the past few contests, and also the mid-contest analysis. I had hoped that Steve Eddins would write one up for this contest, as he seems ideally qualified. > Darren: I'm glad to hear you've found that helpful in the past. I apologize again to the community for not being able to do it this time due to the amount of effort I expended on redoing my auto-submission code. However I will endeavor to do it again in future contests.
From: Amitabh Verma on 7 May 2010 01:14
Hannes: <snip>"On the other side of the equation, there is the "most of the user community" group that would supposedly abandon the contest if they were expected to do more than parameter tweaks. Looking at the same stats page we see a total of 5 participants with more than 200 entries. For each of these 5 participants I can point to a productive change they've applied to the code that improved (as opposed to just overfitted) it. So I don't know who these people who are unable or unwilling to do more than change 0.38 to 0.37 are. Maybe you'd like to give a few examples?"</snip> As a newcomer from my side I did start with studying the code while I looked at certain parameters how they affected but could only do so within a certain capacity due to time constraints. In that respect maybe if the Matlab teams advertised a little description about the upcoming problem statement the prior weekend, it would get the new comers prepared a little better when participating. Based on Alan's scoring formula I tried to look at aspects which would have the most impact and started with getting the complexity down. My 2 entries were aimed at getting the complexity down to 10. Make it Faster Righter.. Simpler http://www.mathworks.com/matlabcentral/contest/contests/2/submissions/2498 10 Pointer http://www.mathworks.com/matlabcentral/contest/contests/2/submissions/2963 I personally am not happy when I look at the number of entries I submitted. A majority of them were actually due to my own fault of interpreting the Longevity contest the wrong way. I was trumping my own entries diminishing my own lead. And not to mention sleep deprivation was making me talk to my own entries as you can see from some of the titles ;) Regarding the bots I'd like to point to the following lines from the contest page: - Out of consideration for everyone participating in the contest, we ask that you not abuse the system. - Tuning the entry to the contest test suite via tweak bombing or other techniques is permitted, but we ask that you not overwhelm the queue. - Max 10 submits per player in 10 minutes: This is another request from the community do minimize the queue flooding especially towards the end of the contest. If anyone updates their queue flooding scripts for the new contest, after each 10 files, you will get an error message. - On the “1 user = 1 login id”, this is tied to your MathWorks.com login. So one login = 1 contest user. http://www.mathworks.com/matlabcentral/contest/contests/2/rules http://blogs.mathworks.com/contest/2010/04/28/new-contest-website-features/#comments Of course, these don't mention explicitly anything but I think we still do understand what they are trying to say. Finally, regarding alternate algorithms that are not favored by the tweaking community in the day and suffers due to the combined optimization as someone mentioned. An example I thought being try05 by Alfonso Nieto-Castanon http://www.mathworks.com/matlabcentral/contest/contests/2/submissions/838 I had the following suggestion which could run parallel and still use the same scoreboard. At the end of Twilight the top 5-10 contestant be given the chance to form a team with each one as a mentor to their team. If someone has time issues, he/she could delegate a willing successor. The team members contribution can be documented in the code which can be only submitted by the mentor of that team. The advantages would be endless. Newcomers will feel more involved and have a better understanding of the underlying mechanism. This they can then apply to their individual entry. In turn mentor gets the tweaking feedback from the team members and more time to focus on the actual code. The mentor is not obliged to share the final code with the team-members in case there are any trust issues. For their guidance they get the feedback. |