T-Tests in crepe

Preparation

First, navigate to your client folder, go to analyses and open ttest_grid.csv. The file will looks somewhat like this.

var	label	initcont	ctctfreq	`...`
GROUP_1	Value(s) for first group	`2`	`5`
GROUP_2	Value(s) for second group	`ELSE`	`1 \| 2 \| 3 \| 4`
GROUP_1_LABEL	Description of Group 1	Grantees who most frequently initiate contact with funder	Grantees who have contact with funder yearly or less often
GROUP_2_LABEL	Description of Group 2	Grantees who did not have initiate contact most frequently	Grantees who have contact with funder at least a few times a year
GROUP_1_N	Overall first group size
GROUP_2_N	Overall second group size
HEADING	Impact on Grantee Fields and Local Communities
SUBHEADING	Field-Focused Measures
fldimp	Impact on Grantees’ Fields
undrfld	Understanding of Grantees’ Fields
advknow	Advancing Knowledge in the Field
pubpol	Funder’s Effect on Public Policy in Grantees’ Fields
SUBHEADING	Community-Focused Measures
comimp	Impact on Grantees’ Local Communities
undrwr	Understanding of Grantees’ Local Communities
`...`

All values for each variable row are currently blank, but will be filled out as soon as you run the T-tests in crepe.

Add custom questions to T-Tests

If you want to add custom questions to the list of dependent variables for testing, add them under the Custom Questions row. For example:

var	label	initcont	ctctfreq	`...`
`...`	`...`	`...`	`...`	`...`
HEADING	Custom Questions
SUBHEADING	Questions Related to Area A
f1234_a1	Area A - Question 1
f1234_a2	Area A - Question 2
SUBHEADING	Questions Related to Area B
f1234_b1	Area B - Question 1
f1234_b2	Area B - Question 2
f1234_b3	Area B - Question 3
f1234_b4	Area B - Question 4

Add independent variables to T-Tests

If you want to run more T-Tests beyond the standard analyses, add the name of the variables and their corresponding meta-data to the right of the T-Test grid. For example:

var	label	`...`	gender	f1234_subgroup_1	f1234_subgroup_2
GROUP_1	Value(s) for first group	`...`	`1`	`1`	`2 \| 3 \| 4`
GROUP_2	Value(s) for second group	`...`	`2`	`0`	`5 \| 6`
GROUP_1_LABEL	Description of Group 1	`...`	Male grantees	Grantees who subscribed to X	Subgroups 2, 3, and 4
GROUP_2_LABEL	Description of Group 2	`...`	Female grantees Grantees who did not subscribe to X	Subgroups 5 and 6
GROUP_1_N	Overall first group size	`...`
GROUP_2_N	Overall second group size	`...`
HEADING	Impact on Grantee Fields and Local Communities	`...`

Here are a few things to note as you add new columns to ttest_grid.csv:

Grouping values: under rows GROUP_1 and GROUP_2, crepe support 3 types of input:
- Numeric: In the example above, crepe will run T-Tests on those indicating 1 with those indicating 2 for variable f1234_subgroup_1.
- String: It’s also possible to enter names, such as Subgroup Category 1 or Program Area X for a variable, say, f1234_subgroup_label. However, make sure that the f1234_subgroup_label contains those labels. Where possible, using numeric input is preferred.
- Multiple values: To save time for recoding, you may enter multiple values in the cell, separated by the | character. In the example above, for f1234_subgroup_2, crepe will run T-Tests for those indicating 2, 3, and 4, versus those indicating 5 or 6. IMPORTANT: MAKE SURE THERE IS A SPACE BEFORE AND AFTER THE |, or your T-Test results will be incorrect.
Group description: under rows GROUP_1_LABEL and GROUP_2_LABEL, you need to enter a description for each group so anyone knows which groups are being compared in the T-Test report.

Once you have added the custom questions for testing, you’re ready to run the T-Tests

Execution

From crepe interface, use option 7 to view analyses options for crepe

STAN THE MAN
=======================================
Stan is here to help you perform STandard ANalyses and more!

You are currently set to work with the following environment:
     - product: gpr
Please select your options:
     1. Run T-tests for a client using ttest_grid.csv
     2. Run ANOVAs for a client using anova_prep.csv
     3. Generate T-test Report


Enter your options here:

T-Test Output Grid

To obtain the raw outputs from the T-Tests for your client, run 7-1 (e.g., press 1 if you’re already on the menu above) and select your funder. crepe may output the results as follows:

Comparing means of groups created with `curfund`
    GROUP 1: [1]
    GROUP 2: [0]
WARNING: N<10 in fdn_xl.csv where var `curfund` equals `0`
Please check input in your grid_ttest.csv
Skipping T-tests for `curfund` because of N<10 group(s)

Comparing means of groups created with `disceval`
    GROUP 1: [1]
    GROUP 2: [0]
WARNING: N<10 in fdn_xl.csv where var `disceval` equals `1`
Please check input in your grid_ttest.csv
Skipping T-tests for `disceval` because of N<10 group(s)

Comparing means of groups created with `sitevst`
    GROUP 1: [1]
    GROUP 2: [0]
ttest_sitevst.csv saved to /Users/miken/Dropbox (CEP)/Python/le-crepe/gpr/clients/<Client>/analyses/outputs

Comparing means of groups created with `eval_prcs_new`
    GROUP 1: [4]
    GROUP 2: [1, 2, 3]
ttest_eval_prcs_new.csv saved to /Users/miken/Dropbox (CEP)/Python/le-crepe/gpr/clients/<Client>/analyses/outputs

Can't find `ISI_value` in the columns of fdn_xl.csv
Please check your input in ttest_grid.csv. Skipping this T-test.

Comparing means of groups created with `gender`
    GROUP 1: [1]
    GROUP 2: [2]
ttest_gender.csv saved to /Users/miken/Dropbox (CEP)/Python/le-crepe/gpr/clients/<Client>/analyses/outputs

Updated ttest_grid.csv with T-test results for <Client>

Here are a few things to note:

No T-Tests will be run if there are fewer than 10 respondents in a group
You can find more output details in your client folder for a variable <variable> under analyses/outputs/ttest_<variable>.csv
ttest_grid.csv will be updated with new results from all the T-Tests run. When you re-open ttest_grid.csv, the result will now look like this:

var	label	initcont	ctctfreq	`...`
GROUP_1	Value(s) for first group	`2`	`5`
GROUP_2	Value(s) for second group	`ELSE`	`1 \| 2 \| 3 \| 4`
GROUP_1_LABEL	Description of Group 1	Grantees who most frequently initiate contact with funder	Grantees who have contact with funder yearly or less often
GROUP_2_LABEL	Description of Group 2	Grantees who did not have initiate contact most frequently	Grantees who have contact with funder at least a few times a year
GROUP_1_N	Overall first group size	30	40
GROUP_2_N	Overall second group size	17	52
HEADING	Impact on Grantee Fields and Local Communities
SUBHEADING	Field-Focused Measures
fldimp	Impact on Grantees’ Fields
undrfld	Understanding of Grantees’ Fields
advknow	Advancing Knowledge in the Field
pubpol	Funder’s Effect on Public Policy in Grantees’ Fields
SUBHEADING	Community-Focused Measures
comimp	Impact on Grantees’ Local Communities
undrwr	Understanding of Grantees’ Local Communities
`...`

Of course, where there are no significant differences in group means, the cell will be blank. If a number is shown, e.g., 0.77, this indicates that group 1 rates higher than group 2 by 0.77, and that mean difference is significant. If there is an asterisk next to a number, e.g., 0.87*, it means that the difference is signi. ficant at either medium or large effect size.

T-Test Report

While the output grid above is helpful for a quick scan, it is not exactly readable. To help with readability, crepe offers HTML output, allowing readers to browse through multiple outputs on a mini-website. Run 7-3 and select your funder. You should then get a T-Test HTML report in your client folder under analyses with the name ttest_results.html. A T-Test report may look like this.

T-Test Output –> Report Routine

Every time you add a new custom question variable as either independent or dependent variable, you’ll need to re-run 7-1 and 7-3 for your client, so that the output and the HTML report reflect the changes you made to the grid.