Project Manual

1 Stages

There are several stages of the project:

1. identify as many measures as possible that assess externalizing problems and related constructs (and obtain the freely available ones)
   • fill out the Measures tab of the Existing Measures and Items spreadsheet
   • add the instrument (as documents/PDFs) to the Instruments folder
   • add the source reference(s) (as documents/PDFs) to the Source References folder
   • add the scoring manual(s) (as documents/PDFs) to the Scoring folder
2. obtain as many measures as possible that are not freely available
   • i.e., add additional instruments to the Instruments folder
3. identify the scales and subscales for each measure
   • i.e., fill out the Scales and Subscales tab of the Existing Measures and Items spreadsheet
4. identify the items for each scale and subscale for each measure
   • i.e., fill out the Items tab of the Existing Measures and Items spreadsheet
5. identify the facet assessed by each item
   • i.e., fill in the facet column of the Items tab of the Existing Measures and Items spreadsheet
6. select and edit the items to ensure each facet is assessed comprehensively at each age
7. have the focus groups of experts review, refine, and rate the construct relevance of the items at each age
8. select the final item pool for each age
9. recruit participants to test the item pools
10. select items for the final measure at a given age for a given rater type based on testing
11. recruit a nationally representative sample of participants whose informants who will rate them on the measure
12. generate age- and sex-based norms
13. develop a computerized adaptive test
14. develop a computerized scoring system
15. develop translated versions

2 Instructions for all Spreadsheets

• Use Roboto font size 10
• Use an en dash (–; i.e., not a hyphen) to indicate a range:
  • e.g., 1–18 (not 1-18)
  • an en dash is technically correct; in addition, spreadsheets often read 3-7 as March 7th, but they correctly read 3–7

3 Stage 1: Identify Measures

This stage involves 1) identifying as many measures as possible that assess externalizing problems and related constructs and 2) obtaining the freely available ones. To achieve this, conduct a thorough literature search using the suggested search terms (below). As you identify measures, fill out the Measures tab of the Existing Measures and Items spreadsheet and (if the instrument is freely available) add the instrument to the Instruments folder.

3.1 Search Terms

The following are search terms to use for identifting measures that assess externalizing problems and related constructs:

• externalizing
• antisocial
• disruptive
• conduct problems
• callous-unemotional
• aggression
• oppositionality
• inattention
• hyperactivity
• ADHD
• conduct disorder
• oppositional defiant disorder
• antisocial personality disorder

3.2 Fill out the Measures tab

Fill out the Measures tab of the Existing Measures and Items spreadsheet, located here. If there are multiple editions of the same measure, select (and specify) the most recent edition (e.g., Behavior Rating Inventory of Executive Function–2 for the 2nd Edition of the BRIEF).

3.2.1 Data Dictionary

• measureName: the full official measure name

• abbreviation: the (capitalized) abbreviation of the measure name; duplicates are allowed

• measureClass: the broader assessment family/system; if multiple measures are from the same assessment family/system (e.g., both the Child Behavior Checklist and Teacher’s Report Form are part of the Achenbach System of Empirically Based Assessment) provide a short label for the measure class; if the measure is not part of a broader measure class, use the lowercase version of the abbreviation

  • should be lowercase with no punctuation
  • only measures that are part of the same measure class should have the same label for measureClass; if measures from a different measure class have the same label for measureClass, this may have to include a disambuating element (such as the author name or year of publication) so measures from different measure classes are not given the same label

• instrument: the lowercase version of the abbreviation

  • should be lowercase with no punctuation
  • this may have to include a disambuating element (such as the author name or year of publication) so that there are no duplicates in the measure column

• measure: a calculated field that combines the measureClass and the instrument into one field, separated by an underscore; there should be no duplicates (duplicates are highlighted in red); this field is the unique identifier of the measure

  • do not enter this manually; drag the formula from other cells in the column to calculate

• type: the type of measure; one of:

  • questionnaire
  • interview
  • observation
  • other

• rater: the rater for whom the measure was designed to be completed; one of:

  • parent
  • teacher
  • caregiver: caregivers who are not parents or teachers (e.g., other family members or care providers)
  • self
  • peer
  • nurse
  • clinician
  • nurse
  • sibling
  • other: any informant who knows the child well

• ageLowerYears: the lower limit of the ratee’s age range for which the measure was designed to be completed (in years)

• ageUpperYears: the upper limit of the ratee’s age range for which the measure was designed to be completed (in years); if no upper limit is specified in the documentation for the instrument, use the oldest age in the original sample. If the oldest age is not available and the scale was designed to be used without an upper age limit, specify 99.

• ageLowerMonths: the lower limit of the ratee’s age range for which the measure was designed to be completed (in months)—especially relevant for infants/toddlers

  • this is calculated automatically using a formula based on ageLowerYears, so you should not enter this unless the measure specifies ratee’s age range in months rather than years

• ageUpperMonths: the upper limit of the ratee’s age range for which the measure was designed to be completed (in months)—especially relevant for infants/toddlers

  • this is calculated automatically using a formula based on ageUpperYears, so you should not enter this unless the measure specifies ratee’s age range in months rather than years

• responseOptions: the response options for each item

  • Use a semicolon (;) to separate response options
  • For example: 0=Not True (as far as you know); 1=Somewhat or Sometimes True; 2=Very True or Often True

• proprietary: whether the measure is proprietary; one of:

  • yes
  • no

• notes: any notes on the measure

• url: the URL for the measure (i.e., where to access or purchase it); you can include multiple URLs if they would be useful (separate them with a semicolon)

• keyReferences: any key references for the measure, including seminal references, references that include the items, etc.; you can include multiple key references if they would be useful (separate them with a semicolon)

• instrumentPDF: whether or not we have the PDF of the instrument (indicate yes with an x); if we are unable to obtain it, specify why not (e.g., need to order or cannot find)

• sourceRefPDF: whether or not we have the PDF of the source reference (indicate yes with an x); if we are unable to obtain it, specify why not (e.g., need to request or cannot find)

• availableToOrder: whether the instrument is available to order from publisher or instrument creator (indicate yes with an x if instrument is available to order); if we are unable to obtain it, leave blank

3.2.2 How Rows are Sorted

The rows in the Measures tab of the Existing Measures and Items spreadsheet are sorted by the measure column, which sorts first by measureClass and then by instrument. When adding rows for new measures, place the new row in the appropriate location accordingly.

3.2.3 How to Indicate Measures from the Same “Measure Class”

As described in the Data Dictionary, the “measure class” is the broader assessment family or system. Multiple instruments can belong to the same measure class (e.g., both the Child Behavior Checklist and Teacher’s Report Form are part of the Achenbach System of Empirically Based Assessment). If there are multiple measures within the same measure class, use borders to indicate that measures have the same measure class. For instance, the first assessment in a given measure class should have a top border, and the last assessment in a given measure class should have a bottom border.

3.3 Add instruments to the Instruments folder

If the instrument is freely available—i.e., publicly available or provided in a publication—add the instrument to the Instruments folder, located here. Save the PDF using the capitalized abbreviation of the measure name.

3.4 Add source reference(s) to the Source References folder

If the instrument is freely available—i.e., publicly available or provided in a publication—add the source reference(s) to the Source References folder, located here. Save the source reference(s) using the following format, depending on the number of authors:

• one author: “Author, Year, Abbreviation.pdf”
  • e.g.: “Achenbach, 2000, CBCL.pdf”
• two authors: “Author1 & Author2, Year, Abbreviation.pdf”
  • e.g.: “Achenbach & Rescorla, 2000, CBCL.pdf”
• three or more authors: “Author1 et al, Year, Abbreviation.pdf”
  • e.g.: “Achenbach et al, 2000, CBCL.pdf”

3.5 Add scoring manual(s) to the Scoring folder

If the instrument is freely available—i.e., publicly available or provided in a publication—add the scoring manual(s) (as documents/PDFs) to the Scoring folder, located here. Save the PDF using the capitalized abbreviation of the measure name.

4 Stage 2: Obtain Measures

This stage involves obtaining as many measures as possible that are not freely available—e.g., measures that are proprietary (e.g., by purchasing them) or are not provided in publications (e.g., by contacting the authors). After obtaining measures, complete steps of Stage 1 with the newly obtained measures, including:

• add the instrument (as documents/PDFs) to the Instruments folder (described here)
• add the source reference(s) (as documents/PDFs) to the Source References folder (described here)
• add the scoring manual(s) (as documents/PDFs) to the Scoring folder (described here)

5 Stage 3: Identify the Scales and Subscales for Each Measure

This stage involves identifying the scales and subscales for each measure. For a measure, identify the names of the scales and subscales as specified by the authors. Fill out the Scales and Subscales tab of the Existing Measures and Items spreadsheet. Follow the Camel case naming convention when adding the names of the scales and subscales to the Measures table (e.g., “Rule Breaking Behavior” = ruleBreakingBehavior). If a scale or subscale does not have a clear name, use the naming convention to label with 1–3 words that best describe the scale or subscale.

5.1 Data Dictionary

• measure: the unique identifier of the measure; combines the measureClass and the instrument into one field; should be the same value as the measure in the Measures tab of the Existing Measures and Items spreadsheet; see the Data Dictionary for the Measures tab for a detailed description of the column
• higherOrderScale: the higher-order scale(s) (if any) for the measure; a higher-order scale is a scale that subsumes multiple subscales; should be “n/a” if the measure does not have a higher-order scale
• subscale: the subscale(s) of the measure;

6 Stage 4: Identify the Items

This stage involves identifying the items for each scale and subscale for each measure. To do this, fill out the Items tab of the Existing Measures and Items spreadsheet by transcribing the items from the PDF of each measure that is located in the Instruments folder.

6.1 Data Dictionary

• itemWording: the wording of the item
• facet: the facet that is assessed by the item
• reverseScored: whether the item is reverse scored; one of:
  • yes
  • no
• measure: the unique identifier of the measure that the item is from; combines the measureClass and the instrument into one field; should be the same value as the measure in the Measures tab of the Existing Measures and Items spreadsheet; see the Data Dictionary for the Measures tab for a detailed description of the column
• itemNumber: the number of the item on measure1
• higherOrderScale: the higher-order scale of the item on measure; see the Data Dictionary for the Scales and Subscales tab for a detailed description of the column
• itemSortOrder: the sort order (ascending) for an item in a given measure; the sort order should reflect the order that we want the items presented—either based on subscale groupings or based on the order in which they are present in the instrument
• subscale: the subscale of the item on measure; see the Data Dictionary for the Scales and Subscales tab for a detailed description of the column
• sinceSeptember2024: Whether the item was added after September 2024; enter an x to indicate yes.

7 Stage 5: Identify the Facet Assessed by Each item

To prevent multiple people from coding the same scale, write your name in the itemFacet column next to each measure you are working on in ‘Measures’. Using the list of ‘Facets’, identify the facet of each item’s for the scale you have selected to work on here: ‘Items’ You can leave comments in [‘Items’] for yourself to address. If questions come up for the group to discuss or if you have ideas for revising the facets, you can use Slack. Mark an x by any scales you’ve completed (i.e., which have all the item facets identified) in the itemFacet column in ‘Measures’.

---

---