Rediscovering the Obvious

…stumbling in the footsteps of greatness

The 11th hour

with 9 comments

We are a community that values delivering early and often. Releasing incomplete work in order to gather feedback and improve our materials is in our DNA, right? So I fully expected to see a nice, smooth line of submissions entering the Agile 2013 submission system over the two months it was open. Yeah, right!

Graph of cumulative submissions to Agile 2013
Graph of daily submissions to Agile 2013

I get to see a few interesting behaviors in my role on the Agile 2013 program committee, but the most interesting is just how strong student syndrome is in our community. A couple of the more interesting things I noted:

  • The several emails to the program committee asking what time of day a two-month-long window will close for new submissions
  • The several people who potentially lose their ability to submit due to technical issues in the last hours of the submission window
  • 70% of submissions were submitted in the last 7 days
  • 54% of submissions were submitted in the last 2 days!

Sigh.

Written by erwilleke

February 2nd, 2013 at 11:02 am

Posted in Uncategorized

9 Responses to 'The 11th hour'

Subscribe to comments with RSS or TrackBack to 'The 11th hour'.

  1. but the question is: was there a promise or implication of early feedback? Student syndrome is driven, in general, by a lack of a promise of interim feedback. If I submit my paper 1 week early, the professor will not grade it and give me a chance to resubmit. If I submit my talk proposal at the end of week one, will the organizers review my proposal at that point in time and give me feedback that will enable me to revise my proposal to the point of acceptance? Not usually. Is that the case for Agile 2013?

    Jon

    3 Feb 13 at 2:32 pm

  2. The Agile conference does indeed reward early submissions with additional rounds of feedback and opportunities to improve the sessions. I’m going to do a correlation after acceptances to see how evenly the accepted talks are distributed through the submission window. Will about 50% of the accepted sessions come from the last two days of submissions?

    erwilleke

    3 Feb 13 at 7:05 pm

  3. Except that the process of acceptance isn’t Agile, it’s a huge batch at the end. Even if I submit early, I have no idea if my proposal is accepted until months later (early feedback doesn’t help if it still results in a rejection). This is why I prefer conferences with rolling acceptance (we did this for SF Agile Con 2012): as soon as a proposal meets acceptance criteria, it’s in. This will eliminate the “student syndrome”, because if wait to submit, I’ll be “competing” for fewer slots.

    Ted M. Young

    19 Feb 13 at 1:15 pm

  4. Agreed, the rolling acceptance isn’t there (yet?), but the ability to refine off feedback is, and we’ve instructed reviewers more thoroughly in setting clear expectations based on review feedback. (no more “nice session, Reject!” behaviors)

    erwilleke

    19 Feb 13 at 6:27 pm

  5. Enter Parkinson’s law…

    http://en.wikipedia.org/wiki/Parkinson‘s_law

    Just because people are “agile” doesn’t make them immune. Also, very few people “doing” agile are actually practitioners of the values/principles but merely “doers” of the process.

    Derek Neighbors

    18 Mar 13 at 8:50 am

  6. I gained a lot by having the feedback early, even if I am not going to learn about acceptance. I do agree with Ted that we could do something about acceptance.

    Waiting for this long for acceptance doesn’t do us any good. We could accept sessions earlier, and start marketing the conference earlier.

    We did receive several *wonderful* submissions late. WRT to Ted’s comment, I wonder if we had told people that we would accept in a rolling fashion, or if we had a shorter cycle, if that would help?

    I spent many hours reviewing. I would rather not invest that much time reviewing. I would rather help people prepare a great talk or a great paper, if they want the help.

    Amen to Derek’s comment. You can see that in the proposals.

    Johanna Rothman

    18 Mar 13 at 10:16 am

  7. Tying early acceptance to early submissions should definitely influence behavior of the submitters. Let’s put this idea in our memory banks for the Track Chairs and Reviewers retrospective. I’d consider having targets for early acceptance that were conservative in the first year with an eye to increasing acceptance levels if the response is effective.

    Thanks Eric for capturing and sharing the stats on submission timing. Now we have a baseline from which we can measure impact of future adaptations.

    Linda Cook

    29 Mar 13 at 7:44 am

  8. Wondering if there’s any correlation between quality of presentations and timliness. Seems that the assumption is that waiting until the last minute is a bad thing — is it? It may be (especially from the reviewers’ perspective), just would like to challenge it. Some people work very well under pressure (ok, I’m one of them). Not sure how you’d measure quality — perhaps by the rating system after each presentation?

    Carolyn Siria

    4 Apr 13 at 5:37 pm

  9. Nice timing! I just got back from Nashville where the program committee put together the recommendations from the various tracks into a coherent schedule, and earlier tonight I correlated the submission date against the acceptance date to see what impact there would be. And I’ll share the answer in a blog post in about two weeks when the letters go out and I can point out specifics without violating our working agreement of not pre-announcing acceptance :)

    erwilleke

    4 Apr 13 at 6:06 pm

Leave a Reply