Review Process

Summary of the BUCLD abstract review process:

1. We send each abstract to 5 reviewers.
2. Reviewers rate each abstract independently on a scale of 1–7 (double-blind procedure) and optionally submit comments for the authors.
3. We calculate two scores for each abstract: (a) mean raw score, (b) mean z score.
4. We rank each abstract by raw score and z score, and calculate a composite rank.
5. We select about 69 talks, 108 posters, and 12 alternates.

Acceptance rates for recent BUCLDs have ranged from 29% to 49%.

Assignment of abstracts to reviewers:

1. Abstracts are individually assigned to five reviewers by the BUCLD faculty advisors, with the help of an automated computer program, on the basis of information indicated by authors and reviewers. The submitting author of each abstract selects codes for the content area of the abstract, the types of learners represented, and the languages studied. Each reviewer similarly indicates his/her expertise in content areas, types of learners, and languages. Also taken into account are the following criteria:

• Ensure that the reviewer is sufficiently familiar with the content of the abstract.
• Ensure that the reviewer is not unfriendly to the theoretical perspective of the abstract.
• Don’t assign abstracts to reviewers who are colleagues, students, advisors, close friends, or enemies of the authors (insofar as we know this).
• Each reviewer gets between 7 and 20 abstracts.

Rating by external reviewers:

2. Reviewers rate each abstract independently on a scale of 1–7 (double-blind procedure), and optionally submit comments for the authors. Reviewers are asked to use the criteria as outlined in the guidelines section and as appropriate for the abstracts they evaluate. (Note that not all criteria will apply equally well to each abstract.)

Scoring and ranking:

3. We calculate two scores for each abstract:

• mean raw score
• mean z score

RAW SCORE

Z SCORE

4. We rank each abstract by raw score and z score, and calculate a composite rank. Note that only one first-author submission is accepted per person; so if more than one submission by an author is ranked within the accepted range, only the highest ranked one is accepted.

Selection of papers, alternates, and posters:

5. We select 69 abstracts to be presented as papers, 12 as alternates, and 108 as posters.

PAPER SELECTION PROCESS

From the set of abstracts designated as “paper only” or “either paper or poster”:

• We select the top 50 abstracts from the raw score list, and the top 50 abstracts from the z score list. This totals 60-70 abstracts (there is a lot of overlap between the two sets).
• We create a pool of the next 40 abstracts based on composite rank.
• We select abstracts from the pool to complete the program of 69 papers, based as much as possible on composite rank, with the goal of forming coherent sessions.

ALTERNATE SELECTION PROCESS

From the set of abstracts designated as “paper only” or “either paper or poster” which were indicated by the authors as possible alternates:

We select 12 alternate abstracts from the remaining abstracts in the pool, based as much as possible on composite rank, with the goal of getting a good distribution of content areas.

POSTER SELECTION PROCESS

From the set of abstracts designated as “poster only” or “either paper or poster”:

• We eliminate all abstracts already selected as papers.
• We select the top 108 remaining abstracts based on composite rank.

Acceptance rates in recent years:

*Does not include the 12 alternate papers. BUCLD began having posters in 2003.