T-Tests in crepe
Preparation
First, navigate to your client folder, go to analyses
and open ttest_grid.csv
. The file will looks somewhat like this.
var | label | initcont | ctctfreq | ... |
GROUP_1 | Value(s) for first group | 2 |
5 |
|
GROUP_2 | Value(s) for second group | ELSE |
1 | 2 | 3 | 4 |
|
GROUP_1_LABEL | Description of Group 1 | Grantees who most frequently initiate contact with funder | Grantees who have contact with funder yearly or less often | |
GROUP_2_LABEL | Description of Group 2 | Grantees who did not have initiate contact most frequently | Grantees who have contact with funder at least a few times a year | |
GROUP_1_N | Overall first group size | |||
GROUP_2_N | Overall second group size | |||
HEADING | Impact on Grantee Fields and Local Communities | |||
SUBHEADING | Field-Focused Measures | |||
fldimp | Impact on Grantees’ Fields | |||
undrfld | Understanding of Grantees’ Fields | |||
advknow | Advancing Knowledge in the Field | |||
pubpol | Funder’s Effect on Public Policy in Grantees’ Fields | |||
SUBHEADING | Community-Focused Measures | |||
comimp | Impact on Grantees’ Local Communities | |||
undrwr | Understanding of Grantees’ Local Communities | |||
... |
All values for each variable row are currently blank, but will be filled out as soon as you run the T-tests in crepe.
Add custom questions to T-Tests
If you want to add custom questions to the list of dependent variables for testing, add them under the Custom Questions row. For example:
var | label | initcont | ctctfreq | ... |
... |
... |
... |
... |
... |
HEADING | Custom Questions | |||
SUBHEADING | Questions Related to Area A | |||
f1234_a1 | Area A - Question 1 | |||
f1234_a2 | Area A - Question 2 | |||
SUBHEADING | Questions Related to Area B | |||
f1234_b1 | Area B - Question 1 | |||
f1234_b2 | Area B - Question 2 | |||
f1234_b3 | Area B - Question 3 | |||
f1234_b4 | Area B - Question 4 |
Add independent variables to T-Tests
If you want to run more T-Tests beyond the standard analyses, add the name of the variables and their corresponding meta-data to the right of the T-Test grid. For example:
var | label | ... |
gender | f1234_subgroup_1 | f1234_subgroup_2 |
GROUP_1 | Value(s) for first group | ... |
1 |
1 |
2 | 3 | 4 |
GROUP_2 | Value(s) for second group | ... |
2 |
0 |
5 | 6 |
GROUP_1_LABEL | Description of Group 1 | ... |
Male grantees | Grantees who subscribed to X | Subgroups 2, 3, and 4 |
GROUP_2_LABEL | Description of Group 2 | ... |
Female grantees Grantees who did not subscribe to X | Subgroups 5 and 6 | |
GROUP_1_N | Overall first group size | ... |
|||
GROUP_2_N | Overall second group size | ... |
|||
HEADING | Impact on Grantee Fields and Local Communities | ... |
Here are a few things to note as you add new columns to ttest_grid.csv
:
- Grouping values: under rows
GROUP_1
andGROUP_2
, crepe support 3 types of input:- Numeric: In the example above, crepe will run T-Tests on those indicating
1
with those indicating2
for variablef1234_subgroup_1
. - String: It’s also possible to enter names, such as
Subgroup Category 1
orProgram Area X
for a variable, say,f1234_subgroup_label
. However, make sure that thef1234_subgroup_label
contains those labels. Where possible, using numeric input is preferred. - Multiple values: To save time for recoding, you may enter multiple values in the cell, separated by the
|
character. In the example above, forf1234_subgroup_2
, crepe will run T-Tests for those indicating2
,3
, and4
, versus those indicating5
or6
. IMPORTANT: MAKE SURE THERE IS A SPACE BEFORE AND AFTER THE|
, or your T-Test results will be incorrect.
- Numeric: In the example above, crepe will run T-Tests on those indicating
- Group description: under rows
GROUP_1_LABEL
andGROUP_2_LABEL
, you need to enter a description for each group so anyone knows which groups are being compared in the T-Test report.
Once you have added the custom questions for testing, you’re ready to run the T-Tests
Execution
From crepe interface, use option 7
to view analyses options for crepe
STAN THE MAN
=======================================
Stan is here to help you perform STandard ANalyses and more!
You are currently set to work with the following environment:
- product: gpr
Please select your options:
1. Run T-tests for a client using ttest_grid.csv
2. Run ANOVAs for a client using anova_prep.csv
3. Generate T-test Report
Enter your options here:
T-Test Output Grid
To obtain the raw outputs from the T-Tests for your client, run 7-1
(e.g., press 1
if you’re already on the menu above) and select your funder. crepe may output the results as follows:
Comparing means of groups created with `curfund`
GROUP 1: [1]
GROUP 2: [0]
WARNING: N<10 in fdn_xl.csv where var `curfund` equals `0`
Please check input in your grid_ttest.csv
Skipping T-tests for `curfund` because of N<10 group(s)
Comparing means of groups created with `disceval`
GROUP 1: [1]
GROUP 2: [0]
WARNING: N<10 in fdn_xl.csv where var `disceval` equals `1`
Please check input in your grid_ttest.csv
Skipping T-tests for `disceval` because of N<10 group(s)
Comparing means of groups created with `sitevst`
GROUP 1: [1]
GROUP 2: [0]
ttest_sitevst.csv saved to /Users/miken/Dropbox (CEP)/Python/le-crepe/gpr/clients/<Client>/analyses/outputs
Comparing means of groups created with `eval_prcs_new`
GROUP 1: [4]
GROUP 2: [1, 2, 3]
ttest_eval_prcs_new.csv saved to /Users/miken/Dropbox (CEP)/Python/le-crepe/gpr/clients/<Client>/analyses/outputs
Can't find `ISI_value` in the columns of fdn_xl.csv
Please check your input in ttest_grid.csv. Skipping this T-test.
Comparing means of groups created with `gender`
GROUP 1: [1]
GROUP 2: [2]
ttest_gender.csv saved to /Users/miken/Dropbox (CEP)/Python/le-crepe/gpr/clients/<Client>/analyses/outputs
Updated ttest_grid.csv with T-test results for <Client>
Here are a few things to note:
- No T-Tests will be run if there are fewer than 10 respondents in a group
- You can find more output details in your client folder for a variable
<variable>
underanalyses/outputs/ttest_<variable>.csv
ttest_grid.csv
will be updated with new results from all the T-Tests run. When you re-openttest_grid.csv
, the result will now look like this:
var | label | initcont | ctctfreq | ... |
GROUP_1 | Value(s) for first group | 2 |
5 |
|
GROUP_2 | Value(s) for second group | ELSE |
1 | 2 | 3 | 4 |
|
GROUP_1_LABEL | Description of Group 1 | Grantees who most frequently initiate contact with funder | Grantees who have contact with funder yearly or less often | |
GROUP_2_LABEL | Description of Group 2 | Grantees who did not have initiate contact most frequently | Grantees who have contact with funder at least a few times a year | |
GROUP_1_N | Overall first group size | 30 | 40 | |
GROUP_2_N | Overall second group size | 17 | 52 | |
HEADING | Impact on Grantee Fields and Local Communities | |||
SUBHEADING | Field-Focused Measures | |||
fldimp | Impact on Grantees’ Fields | |||
undrfld | Understanding of Grantees’ Fields | |||
advknow | Advancing Knowledge in the Field | |||
pubpol | Funder’s Effect on Public Policy in Grantees’ Fields | |||
SUBHEADING | Community-Focused Measures | |||
comimp | Impact on Grantees’ Local Communities | |||
undrwr | Understanding of Grantees’ Local Communities | |||
... |
Of course, where there are no significant differences in group means, the cell will be blank. If a number is shown, e.g., 0.77
, this indicates that group 1 rates higher than group 2 by 0.77, and that mean difference is significant. If there is an asterisk next to a number, e.g., 0.87*
, it means that the difference is signi. ficant at either medium or large effect size.
T-Test Report
While the output grid above is helpful for a quick scan, it is not exactly readable. To help with readability, crepe offers HTML output, allowing readers to browse through multiple outputs on a mini-website. Run 7-3
and select your funder. You should then get a T-Test HTML report in your client folder under analyses
with the name ttest_results.html
. A T-Test report may look like this.
T-Test Output –> Report Routine
Every time you add a new custom question variable as either independent or dependent variable, you’ll need to re-run 7-1
and 7-3
for your client, so that the output and the HTML report reflect the changes you made to the grid.